diff --git a/.gitattributes b/.gitattributes index 54128ed2cf3c8c476403b3575721cb314e77d315..ae17177b2357d1eb491e0c24e1695a4df552a3cb 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,37 +1,3 @@ -*.7z filter=lfs diff=lfs merge=lfs -text -*.arrow filter=lfs diff=lfs merge=lfs -text -*.bin filter=lfs diff=lfs merge=lfs -text -*.bz2 filter=lfs diff=lfs merge=lfs -text -*.ckpt filter=lfs diff=lfs merge=lfs -text -*.ftz filter=lfs diff=lfs merge=lfs -text -*.gz filter=lfs diff=lfs merge=lfs -text -*.h5 filter=lfs diff=lfs merge=lfs -text -*.joblib filter=lfs diff=lfs merge=lfs -text -*.lfs.* filter=lfs diff=lfs merge=lfs -text -*.mlmodel filter=lfs diff=lfs merge=lfs -text -*.model filter=lfs diff=lfs merge=lfs -text -*.msgpack filter=lfs diff=lfs merge=lfs -text -*.npy filter=lfs diff=lfs merge=lfs -text -*.npz filter=lfs diff=lfs merge=lfs -text -*.onnx filter=lfs diff=lfs merge=lfs -text -*.ot filter=lfs diff=lfs merge=lfs -text -*.parquet filter=lfs diff=lfs merge=lfs -text -*.pb filter=lfs diff=lfs merge=lfs -text -*.pickle filter=lfs diff=lfs merge=lfs -text -*.pkl filter=lfs diff=lfs merge=lfs -text -*.pt filter=lfs diff=lfs merge=lfs -text -*.pth filter=lfs diff=lfs merge=lfs -text -*.rar filter=lfs diff=lfs merge=lfs -text -*.safetensors filter=lfs diff=lfs merge=lfs -text -saved_model/**/* filter=lfs diff=lfs merge=lfs -text -*.tar.* filter=lfs diff=lfs merge=lfs -text -*.tar filter=lfs diff=lfs merge=lfs -text -*.tflite filter=lfs diff=lfs merge=lfs -text -*.tgz filter=lfs diff=lfs merge=lfs -text -*.wasm filter=lfs diff=lfs merge=lfs -text -*.xz filter=lfs diff=lfs merge=lfs -text -*.zip filter=lfs diff=lfs merge=lfs -text -*.zst filter=lfs diff=lfs merge=lfs -text -*tfevents* filter=lfs diff=lfs merge=lfs -text -*.cbm filter=lfs diff=lfs merge=lfs -text -Dataset.xlsx filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md index ba6e57b76e26a76d46c68da765c7b6471c720971..36b85c1bbc36e7fbf2bd53966e1586c840ffeac0 100644 --- a/README.md +++ b/README.md @@ -1,127 +1,75 @@ ---- -title: MLATE -emoji: πŸ₯Ό -colorFrom: red -colorTo: indigo -sdk: streamlit -sdk_version: 1.44.1 -python_version: "3.12" -app_file: app.py -pinned: false -license: cc-by-nc-4.0 -short_description: ML Applications in Tissue Engineering ---- - - -# πŸ‘¬ MLATE V3: Multi-Tissue Scaffold Prediction Platform - -**MLATE V3** is a fully integrated, machine learning-powered platform for predicting, optimizing, and generating detailed fabrication procedures for 3D-(bio)printed scaffolds in tissue engineering. This app enables researchers to input a wide range of biomaterials, cell lines, and printing parameters, and receive optimized scaffold compositions along with step-by-step printing instructions generated via Google Gemini. - -> πŸ“„ *Rafieyan et al. (preprint, 2025). MLATE V3: A fully integrated Multi-Tissue, machine learning platform for prediction, optimization and generating procedures for fabricating 3D-(bio)printing scaffolds for tissue engineering* - ---- - -## πŸš€ Features - -- πŸ”¬ Predict scaffold quality based on printability and cell response -- πŸ§ͺ Optimize biomaterial concentrations, cell densities, and printing parameters using Optuna -- 🧠 Powered by two fine-tuned **CatBoostClassifier** models -- πŸ“‹ Automatically generates fabrication protocols with Gemini API -- πŸ” Enforces safe defaults and intelligent UI input validation -- 🧱 Uses a real-world, curated dataset of **2847 samples** across **multiple tissues and cell lines** - ---- - -## πŸ“‚ Dataset - -This project includes a publicly available dataset (`Dataset.xlsx`) containing: -- 123 biomaterials -- 175 cell lines -- 7 printing parameters -- Scaffold performance labels - -The dataset is available in the [Files and Versions](https://huggingface.co/spaces/your-username/your-space-name/blob/main/Dataset.xlsx) tab of this Space. - -You may also optionally add this to Hugging Face Datasets for broader access. - ---- - -## βš™οΈ How It Works - -1. **Input**: User selects biomaterials, cell line, and printing parameters with min/max/step values -2. **Optimization**: Optuna runs 50 trials to maximize predicted scaffold quality (WSSQ) -3. **Prediction**: - - Two CatBoost models are used to predict: - - `Printability` (3-class) - - `Cell Response` (5-class) - - Probabilistic predictions are mapped to expected scores -4. **Scaffold Quality**: A weighted combination of printability and cell response -5. **Procedure Generation**: A Gemini API prompt generates custom step-by-step fabrication instructions - ---- - -## πŸ’» Running Locally - -Clone the repo and install dependencies: - -```bash -git clone https://huggingface.co/spaces/your-username/MLATE-V3 -cd MLATE-V3 - -# Create virtual environment (optional) -python -m venv venv -source venv/bin/activate # or venv\Scripts\activate on Windows - -# Install dependencies -pip install -r requirements.txt - -# Set your Gemini API key -export GEMINI_API_KEY=your_key_here # or set in .env - -# Run the app -streamlit run app.py -``` - ---- - -## πŸ“œ License - -This project is licensed under the **Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)**. - -You are free to: -- Share and adapt the code -- Use the dataset for academic research - -But: -- **Commercial use is prohibited** -- **Citation is required** (see below) - ---- - -## πŸ“š Citation - -If you use MLATE V3 or its dataset in your research, please cite: - -> Rafieyan et al. (preprint, 2025). -> *MLATE V3: A fully integrated Multi-Tissue, machine learning platform for prediction, optimization and generating procedures for fabricating 3D-(bio)printing scaffolds for tissue engineering* -> *(Preprint link to be added after publication)* - -BibTeX: -```bibtex -@article{rafieyan2025mlate, - author = {Rafieyan, Saeed and others}, - title = {MLATE V3: A fully integrated Multi-Tissue, machine learning platform for prediction, optimization and generating procedures for fabricating 3D-(bio)printing scaffolds for tissue engineering}, - journal = {Preprint}, - year = {2025} -} -``` - ---- - -## ⚠️ Disclaimer - -This tool is intended for **research and academic use only**. While we strive for accuracy, the predictions and fabrication procedures are generated using machine learning and language models and may contain errors or inconsistencies. The authors are **not responsible for any unintended consequences** arising from use of this tool in experimental or clinical settings. - ---- - -Developed by [Saeed Rafieyan](https://sraf.ir) +--- +title: MLATE V3 +emoji: 🧬 +colorFrom: blue +colorTo: green +sdk: streamlit +sdk_version: 1.58.0 +app_file: app.py +pinned: false +license: mit +short_description: Optimises 3D-printed and bioprinted scaffolds +--- + +# MLATE V3 + +Predicts **printability** and **cell response** for 3D-printed and bioprinted +scaffolds, searches the formulation space for the composition and printing +conditions that maximise a combined quality score, and drafts a bench protocol +for the result. + +Models were trained on 2,646 scaffold records extracted from the literature. +Predictions are decision support: they narrow the experimental search space and +do not replace experimental validation. + +## Using it + +1. Enter a range for each biomaterial you can work with, and for the printing + parameters your equipment allows. The optimiser searches inside those ranges. +2. Choose a cell line, or `NoCellCultured` for acellular printing. +3. Set the cell-response weight. Printability takes the remainder. +4. Run the optimisation. +5. Optionally generate a fabrication protocol. This step needs a free + [OpenRouter](https://openrouter.ai) key; nothing else does. + +## What the score means + +WSSQ combines the two predicted outcomes through two conjunctive means, so a +scaffold cannot score well by excelling at one objective and failing the other. +Acellular formulations are scored on printability alone rather than penalised +for a biological outcome that does not apply to them. + +## Models + +All three model families are offered for each target, ranked by weighted F1 on +the held-out test partition and refitted on the complete dataset. On a GPU host +the menu opens on the best model overall; on a CPU host it opens on the best +conventional classifier, because the in-context foundation models re-read all +2,646 training records on every pass and turn a search of seconds into one of +minutes. They remain selectable, with their cost stated beside the menu. + +The random-split artefacts are served, this being the interpolation regime in +which the tool is used: adjusting a concentration, substituting a cell line, or +moving a pressure within observed ranges. The study-grouped models are the +conservative estimate for an unseen laboratory and are reported in the paper. + +## Deploying your own copy + +From a checkout of the repository: + +``` +python 06_webapp/export_deployment.py # fit and export the artefacts +python 06_webapp/build_app_data.py # vocabularies, tables, corpus subset +python 06_webapp/deploy_to_hf.py # stage and check, no upload +python 06_webapp/deploy_to_hf.py --push # upload, after `hf auth login` +``` + +`deploy_to_hf.py` assembles a self-contained tree under `deploy/space/`, +verifies that it imports and finds its models, and uploads it. Model artefacts +are not in this directory by default; they are added by that script. + +## Citation + +Rafieyan *et al.*, *MLATE V3: An Open-Source Cross-Tissue AI Framework for +Data-Driven Optimization of 3D-Printed and Bioprinted Scaffolds*. +Code and dataset: https://github.com/saeedrafieyan/mlate diff --git a/app.py b/app.py index 73dfe8ccbed737b5a23309d6a7cf871490d1f8ed..60c34f9b87e51ce7d92a808475f3aa2913fb4ae3 100644 --- a/app.py +++ b/app.py @@ -1,1012 +1,633 @@ -import os -from pathlib import Path -import streamlit as st -import pandas as pd -import joblib -import optuna -import numpy as np - -from google import genai -from google.genai import types - -from biomaterials import BIOMATERIAL_OPTIONS -from cell_lines import CELL_LINE_OPTIONS - -_original_number_input = st.number_input - -def safe_number_input(label, **kwargs): - """ - Clamp `value` into [min_value, max_value] and warn if we had to adjust. - Then call the real st.number_input with the clamped default. - """ - min_value = kwargs.get("min_value", float("-inf")) - max_value = kwargs.get("max_value", float("inf")) - value = kwargs.get("value", min_value) - clamped = min(max(value, min_value), max_value) - if clamped != value: - st.warning( - f"⚠️ Default for β€œ{label}” ({value}) was outside " - f"[{min_value}, {max_value}]; using {clamped} instead." - ) - kwargs["value"] = clamped - return _original_number_input(label, **kwargs) - -st.number_input = safe_number_input - -APP_DIR = Path(__file__).resolve().parent -MODEL_ROOT = APP_DIR / "models" -PREPROCESSOR_DIR = MODEL_ROOT / "preprocessors" - -MODEL_TASKS = { - "printability": { - "folder": MODEL_ROOT / "printability", - "prefix": "Printability_", - "label_encoder": "label_encoder_printability.pkl", - }, - "cell_response": { - "folder": MODEL_ROOT / "cell response", - "prefix": "Cell_Response_", - "label_encoder": "label_encoder_cell_response.pkl", - }, -} - -DL_MODEL_CONFIGS = { - ("printability", "ResNet"): {"n_layers": 6, "hidden_dim": 302, "dropout": 0.190106, "activation_func": "relu"}, - ("printability", "MLP"): {"n_layers": 3, "hidden_dim": 367, "dropout": 0.169472, "activation_func": "tanh"}, - ("printability", "1D_CNN"): {"n_layers": 6, "hidden_dim": 287, "dropout": 0.233072, "activation_func": "relu"}, - ("printability", "FT_Transformer"): {"n_layers": 3, "hidden_dim": 437, "dropout": 0.324095, "activation_func": "GELU"}, - ("printability", "TabNet_Lite"): {"n_layers": 4, "hidden_dim": 283, "dropout": 0.309445, "activation_func": "relu"}, - ("printability", "NODE_Lite"): {"n_layers": 5, "hidden_dim": 289, "dropout": 0.145481, "activation_func": "SELU"}, - ("cell_response", "ResNet"): {"n_layers": 4, "hidden_dim": 269, "dropout": 0.229224, "activation_func": "tanh"}, - ("cell_response", "MLP"): {"n_layers": 5, "hidden_dim": 238, "dropout": 0.256294, "activation_func": "ELU"}, - ("cell_response", "1D_CNN"): {"n_layers": 6, "hidden_dim": 134, "dropout": 0.158794, "activation_func": "SiLU"}, - ("cell_response", "FT_Transformer"): {"n_layers": 5, "hidden_dim": 395, "dropout": 0.185155, "activation_func": "SiLU"}, - ("cell_response", "TabNet_Lite"): {"n_layers": 5, "hidden_dim": 342, "dropout": 0.103486, "activation_func": "SiLU"}, - ("cell_response", "NODE_Lite"): {"n_layers": 2, "hidden_dim": 127, "dropout": 0.328308, "activation_func": "SiLU"}, -} - -MODEL_RANKINGS = { - "printability": [ - "HistGradientBoosting", "TabPFN 2.6", "Bagging", "GradientBoosting", "TabICL v2", - "XGBoost", "LightGBM", "KNeighbors", "LabelPropagation", "ExtraTrees", "MLP", - "LabelSpreading", "TabNet Lite", "MLP DL", "ResNet", "1D CNN", "DecisionTree", - "NODE Lite", "LinearSVC", "LDA", "PassiveAggressive", "CalibratedClassifierCV", - "LogisticRegression", "RidgeClassifier", "FT Transformer", "Perceptron", - "MultinomialNB", "RadiusNeighbors", "ComplementNB", "NuSVC", "AdaBoost", - "SGD", "BernoulliNB", "GaussianNB", "QDA", "RandomForest", "ExtraTreeClassifier", - "DummyClassifier", - ], - "cell_response": [ - "HistGradientBoosting", "TabICL v2", "TabPFN 2.6", "XGBoost", "Bagging", - "KNeighbors", "LabelPropagation", "LabelSpreading", "LightGBM", "FT Transformer", - "CalibratedClassifierCV", "ResNet", "NODE Lite", "1D CNN", "MLP DL", "LDA", - "NuSVC", "MultinomialNB", "LogisticRegression", "LinearSVC", "Perceptron", - "QDA", "RidgeClassifier", "ExtraTrees", "ExtraTreeClassifier", "AdaBoost", - "DecisionTree", "RandomForest", "GradientBoosting", "SGD", "MLP", - "PassiveAggressive", "ComplementNB", "TabNet Lite", "BernoulliNB", - "RadiusNeighbors", "DummyClassifier", "GaussianNB", - ], -} - -PERFORMANCE_GUIDE = { - "printability": """ -| Rank | Model | Framework | Accuracy | F1 | AUC | MCC | -|---:|---|---|---:|---:|---:|---:| -| 1 | HistGradientBoosting | Machine Learning | 0.80 | 0.80 | 0.94 | 0.69 | -| 2 | TabPFN 2.6 | Deep Learning / Transformer | 0.80 | 0.80 | 0.94 | 0.69 | -| 3 | Bagging | Machine Learning | 0.79 | 0.79 | 0.93 | 0.67 | -| 4 | GradientBoosting | Machine Learning | 0.79 | 0.79 | 0.93 | 0.66 | -| 5 | TabICL v2 | Deep Learning / Transformer | 0.79 | 0.79 | 0.94 | 0.68 | -| 6 | XGBoost | Machine Learning | 0.78 | 0.78 | 0.93 | 0.66 | -| 7 | LightGBM | Machine Learning | 0.77 | 0.77 | 0.93 | 0.64 | -| 8 | KNeighbors | Machine Learning | 0.77 | 0.77 | 0.90 | 0.64 | -| 9 | LabelPropagation | Machine Learning | 0.77 | 0.77 | 0.82 | 0.64 | -| 10 | ExtraTrees | Machine Learning | 0.77 | 0.77 | 0.92 | 0.63 | -""", - "cell_response": """ -| Rank | Model | Framework | Accuracy | F1 | AUC | MCC | -|---:|---|---|---:|---:|---:|---:| -| 1 | HistGradientBoosting | Machine Learning | 0.81 | 0.81 | 0.96 | 0.67 | -| 2 | TabICL v2 | Deep Learning / Transformer | 0.79 | 0.79 | 0.97 | 0.65 | -| 3 | TabPFN 2.6 | Deep Learning / Transformer | 0.78 | 0.78 | 0.96 | 0.63 | -| 4 | XGBoost | Machine Learning | 0.79 | 0.77 | 0.96 | 0.65 | -| 5 | Bagging | Machine Learning | 0.77 | 0.77 | 0.95 | 0.62 | -| 6 | KNeighbors | Machine Learning | 0.77 | 0.77 | 0.93 | 0.61 | -| 7 | LabelPropagation | Machine Learning | 0.77 | 0.77 | 0.93 | 0.61 | -| 8 | LabelSpreading | Machine Learning | 0.78 | 0.76 | 0.93 | 0.61 | -| 9 | LightGBM | Machine Learning | 0.78 | 0.76 | 0.96 | 0.64 | -| 10 | FT Transformer | Deep Learning / Transformer | 0.76 | 0.76 | 0.94 | 0.61 | -""", -} - -PERFORMANCE_GUIDE_FULL = { - "printability": """ -# Printability - Merged Performance Summary - -This file contains the aggregated and benchmarked results of traditional Machine Learning (ML) and Deep Learning / Transformer architectures for predicting **Printability**, sorted hierarchically by **F1 Score** and **Accuracy**. - -| Rank | Model | Framework | Accuracy | Precision | Recall | F1 | AUC | MCC | Kappa | -|---:|---|---|---:|---:|---:|---:|---:|---:|---:| -| 1 | HistGradientBoosting | Machine Learning | 0.80 | 0.80 | 0.80 | 0.80 | 0.94 | 0.69 | 0.69 | -| 2 | TabPFN 2.6 | Deep Learning / Transformer | 0.80 | 0.81 | 0.80 | 0.80 | 0.94 | 0.69 | 0.68 | -| 3 | Bagging | Machine Learning | 0.79 | 0.79 | 0.79 | 0.79 | 0.93 | 0.67 | 0.67 | -| 4 | GradientBoosting | Machine Learning | 0.79 | 0.79 | 0.79 | 0.79 | 0.93 | 0.66 | 0.66 | -| 5 | TabICL v2 | Deep Learning / Transformer | 0.79 | 0.81 | 0.79 | 0.79 | 0.94 | 0.68 | 0.67 | -| 6 | XGBoost | Machine Learning | 0.78 | 0.79 | 0.78 | 0.78 | 0.93 | 0.66 | 0.65 | -| 7 | LightGBM | Machine Learning | 0.77 | 0.78 | 0.77 | 0.77 | 0.93 | 0.64 | 0.64 | -| 8 | KNeighbors | Machine Learning | 0.77 | 0.78 | 0.77 | 0.77 | 0.90 | 0.64 | 0.64 | -| 9 | LabelPropagation | Machine Learning | 0.77 | 0.77 | 0.77 | 0.77 | 0.82 | 0.64 | 0.63 | -| 10 | ExtraTrees | Machine Learning | 0.77 | 0.77 | 0.77 | 0.77 | 0.92 | 0.63 | 0.63 | -| 11 | MLP | Machine Learning | 0.76 | 0.76 | 0.76 | 0.76 | 0.90 | 0.63 | 0.63 | -| 12 | LabelSpreading | Machine Learning | 0.76 | 0.76 | 0.76 | 0.76 | 0.90 | 0.62 | 0.62 | -| 13 | TabNet Lite | Deep Learning / Transformer | 0.74 | 0.74 | 0.74 | 0.74 | 0.91 | 0.59 | 0.59 | -| 14 | MLP (DL) | Deep Learning / Transformer | 0.73 | 0.73 | 0.73 | 0.73 | 0.91 | 0.58 | 0.58 | -| 15 | ResNet | Deep Learning / Transformer | 0.73 | 0.73 | 0.73 | 0.73 | 0.91 | 0.57 | 0.57 | -| 16 | 1D CNN | Deep Learning / Transformer | 0.71 | 0.73 | 0.71 | 0.72 | 0.89 | 0.56 | 0.56 | -| 17 | DecisionTree | Machine Learning | 0.71 | 0.72 | 0.71 | 0.71 | 0.89 | 0.54 | 0.54 | -| 18 | NODE Lite | Deep Learning / Transformer | 0.69 | 0.70 | 0.69 | 0.70 | 0.90 | 0.52 | 0.52 | -| 19 | LinearSVC | Machine Learning | 0.69 | 0.68 | 0.69 | 0.68 | 0.85 | 0.49 | 0.49 | -| 20 | LDA | Machine Learning | 0.68 | 0.68 | 0.68 | 0.68 | 0.84 | 0.49 | 0.48 | -| 21 | PassiveAggressive | Machine Learning | 0.69 | 0.68 | 0.69 | 0.67 | 0.85 | 0.48 | 0.48 | -| 22 | CalibratedClassifierCV | Machine Learning | 0.68 | 0.68 | 0.68 | 0.67 | 0.85 | 0.48 | 0.48 | -| 23 | LogisticRegression | Machine Learning | 0.68 | 0.67 | 0.68 | 0.67 | 0.85 | 0.47 | 0.47 | -| 24 | RidgeClassifier | Machine Learning | 0.68 | 0.67 | 0.68 | 0.66 | 0.84 | 0.46 | 0.46 | -| 25 | FT Transformer | Deep Learning / Transformer | 0.66 | 0.65 | 0.66 | 0.65 | 0.87 | 0.45 | 0.45 | -| 26 | Perceptron | Machine Learning | 0.67 | 0.68 | 0.67 | 0.64 | 0.84 | 0.45 | 0.42 | -| 27 | MultinomialNB | Machine Learning | 0.64 | 0.69 | 0.64 | 0.64 | 0.83 | 0.45 | 0.43 | -| 28 | RadiusNeighbors | Machine Learning | 0.66 | 0.70 | 0.66 | 0.63 | 0.87 | 0.43 | 0.39 | -| 29 | ComplementNB | Machine Learning | 0.63 | 0.67 | 0.63 | 0.63 | 0.82 | 0.44 | 0.43 | -| 30 | NuSVC | Machine Learning | 0.60 | 0.69 | 0.60 | 0.61 | 0.84 | 0.45 | 0.42 | -| 31 | AdaBoost | Machine Learning | 0.62 | 0.59 | 0.62 | 0.58 | 0.80 | 0.38 | 0.37 | -| 32 | SGD | Machine Learning | 0.63 | 0.59 | 0.63 | 0.57 | 0.82 | 0.36 | 0.35 | -| 33 | BernoulliNB | Machine Learning | 0.59 | 0.62 | 0.59 | 0.55 | 0.77 | 0.32 | 0.31 | -| 34 | GaussianNB | Machine Learning | 0.33 | 0.70 | 0.33 | 0.36 | 0.74 | 0.25 | 0.19 | -| 35 | QDA | Machine Learning | 0.52 | 0.27 | 0.52 | 0.35 | 0.80 | 0.00 | 0.00 | -| 36 | RandomForest | Machine Learning | 0.52 | 0.27 | 0.52 | 0.35 | 0.78 | 0.00 | 0.00 | -| 37 | ExtraTreeClassifier | Machine Learning | 0.52 | 0.27 | 0.52 | 0.35 | 0.50 | 0.00 | 0.00 | -| 38 | DummyClassifier | Machine Learning | 0.52 | 0.27 | 0.52 | 0.35 | 0.50 | 0.00 | 0.00 | -""", - "cell_response": """ -# Cell Response - Merged Performance Summary - -This file contains the aggregated and benchmarked results of traditional Machine Learning (ML) and Deep Learning / Transformer architectures for predicting **Cell Response**, sorted hierarchically by **F1 Score** and **Accuracy**. - -| Rank | Model | Framework | Accuracy | Precision | Recall | F1 | AUC | MCC | Kappa | -|---:|---|---|---:|---:|---:|---:|---:|---:|---:| -| 1 | HistGradientBoosting | Machine Learning | 0.81 | 0.81 | 0.81 | 0.81 | 0.96 | 0.67 | 0.67 | -| 2 | TabICL v2 | Deep Learning / Transformer | 0.79 | 0.79 | 0.79 | 0.79 | 0.97 | 0.65 | 0.65 | -| 3 | TabPFN 2.6 | Deep Learning / Transformer | 0.78 | 0.78 | 0.78 | 0.78 | 0.96 | 0.63 | 0.63 | -| 4 | XGBoost | Machine Learning | 0.79 | 0.76 | 0.79 | 0.77 | 0.96 | 0.65 | 0.64 | -| 5 | Bagging | Machine Learning | 0.77 | 0.78 | 0.77 | 0.77 | 0.95 | 0.62 | 0.62 | -| 6 | KNeighbors | Machine Learning | 0.77 | 0.77 | 0.77 | 0.77 | 0.93 | 0.61 | 0.61 | -| 7 | LabelPropagation | Machine Learning | 0.77 | 0.77 | 0.77 | 0.77 | 0.93 | 0.61 | 0.61 | -| 8 | LabelSpreading | Machine Learning | 0.78 | 0.76 | 0.78 | 0.76 | 0.93 | 0.61 | 0.61 | -| 9 | LightGBM | Machine Learning | 0.78 | 0.79 | 0.78 | 0.76 | 0.96 | 0.64 | 0.63 | -| 10 | FT Transformer | Deep Learning / Transformer | 0.76 | 0.79 | 0.76 | 0.76 | 0.94 | 0.61 | 0.60 | -| 11 | CalibratedClassifierCV | Machine Learning | 0.76 | 0.76 | 0.76 | 0.75 | 0.94 | 0.59 | 0.59 | -| 12 | ResNet | Deep Learning / Transformer | 0.76 | 0.76 | 0.76 | 0.75 | 0.94 | 0.59 | 0.59 | -| 13 | NODE Lite | Deep Learning / Transformer | 0.76 | 0.75 | 0.76 | 0.75 | 0.95 | 0.59 | 0.59 | -| 14 | 1D CNN | Deep Learning / Transformer | 0.75 | 0.76 | 0.75 | 0.74 | 0.95 | 0.59 | 0.59 | -| 15 | MLP (DL) | Deep Learning / Transformer | 0.75 | 0.75 | 0.75 | 0.74 | 0.95 | 0.57 | 0.57 | -| 16 | LDA | Machine Learning | 0.75 | 0.72 | 0.75 | 0.73 | 0.94 | 0.55 | 0.55 | -| 17 | NuSVC | Machine Learning | 0.74 | 0.73 | 0.74 | 0.73 | 0.95 | 0.55 | 0.55 | -| 18 | MultinomialNB | Machine Learning | 0.74 | 0.72 | 0.74 | 0.73 | 0.95 | 0.55 | 0.55 | -| 19 | LogisticRegression | Machine Learning | 0.74 | 0.70 | 0.74 | 0.71 | 0.95 | 0.54 | 0.54 | -| 20 | LinearSVC | Machine Learning | 0.74 | 0.71 | 0.74 | 0.70 | 0.94 | 0.53 | 0.52 | -| 21 | Perceptron | Machine Learning | 0.74 | 0.68 | 0.74 | 0.70 | 0.94 | 0.53 | 0.52 | -| 22 | QDA | Machine Learning | 0.74 | 0.68 | 0.74 | 0.70 | 0.94 | 0.57 | 0.55 | -| 23 | RidgeClassifier | Machine Learning | 0.73 | 0.70 | 0.73 | 0.70 | 0.94 | 0.52 | 0.51 | -| 24 | ExtraTrees | Machine Learning | 0.74 | 0.71 | 0.74 | 0.68 | 0.95 | 0.55 | 0.52 | -| 25 | ExtraTreeClassifier | Machine Learning | 0.73 | 0.67 | 0.73 | 0.68 | 0.91 | 0.54 | 0.52 | -| 26 | AdaBoost | Machine Learning | 0.74 | 0.65 | 0.74 | 0.67 | 0.91 | 0.59 | 0.55 | -| 27 | DecisionTree | Machine Learning | 0.74 | 0.65 | 0.74 | 0.67 | 0.92 | 0.59 | 0.55 | -| 28 | RandomForest | Machine Learning | 0.74 | 0.65 | 0.74 | 0.67 | 0.92 | 0.59 | 0.55 | -| 29 | GradientBoosting | Machine Learning | 0.74 | 0.65 | 0.74 | 0.67 | 0.92 | 0.59 | 0.55 | -| 30 | SGD | Machine Learning | 0.71 | 0.65 | 0.71 | 0.66 | 0.92 | 0.49 | 0.47 | -| 31 | MLP | Machine Learning | 0.73 | 0.61 | 0.73 | 0.65 | 0.93 | 0.55 | 0.52 | -| 32 | PassiveAggressive | Machine Learning | 0.73 | 0.65 | 0.73 | 0.65 | 0.92 | 0.52 | 0.49 | -| 33 | ComplementNB | Machine Learning | 0.72 | 0.65 | 0.72 | 0.65 | 0.93 | 0.49 | 0.47 | -| 34 | TabNet Lite | Deep Learning / Transformer | 0.70 | 0.67 | 0.70 | 0.65 | 0.93 | 0.48 | 0.46 | -| 35 | BernoulliNB | Machine Learning | 0.71 | 0.60 | 0.71 | 0.63 | 0.93 | 0.48 | 0.45 | -| 36 | RadiusNeighbors | Machine Learning | 0.60 | 0.36 | 0.60 | 0.45 | 0.50 | 0.00 | 0.00 | -| 37 | DummyClassifier | Machine Learning | 0.60 | 0.36 | 0.60 | 0.45 | 0.50 | 0.00 | 0.00 | -| 38 | GaussianNB | Machine Learning | 0.28 | 0.69 | 0.28 | 0.34 | 0.84 | 0.18 | 0.14 | -""", -} - -GEMINI_MODELS = [ - "gemini-3.5-flash", - "gemini-3.1-flash-lite", - "gemini-3.1-pro-preview", - "gemini-3.1-flash-lite-preview", - "gemini-3-flash-preview", - "gemini-2.5-pro", - "gemini-2.5-flash", - "gemini-2.5-flash-lite", -] - -def scaffold_quality_combined(printability, cell_response, - weight_printability=0.3, weight_cell_response=0.7): - """ - Calculates the Weighted Scaffold Synthesis Quality (WSSQ). - """ - if printability == 0: - return 0.0 - - # Normalization - norm_p = printability / 3.0 - - # If cell_response is 1 (minimum), avoid division by zero in harmonic mean - if cell_response <= 1: - return 100 * norm_p - - norm_c = (cell_response - 1) / 4.0 - - # Weighted Harmonic Mean - hm = (weight_printability + weight_cell_response) / ( - (weight_printability / norm_p) + - (weight_cell_response / norm_c) - ) - - # Weighted Multiplicative Component - mc = (norm_p**weight_printability) * (norm_c**weight_cell_response) - - return 100 * ((hm + mc) / 2.0) - -PRINT_PARAM_NAMES = [ - "Physical Crosslinking Duration (s)", - "Photo Crosslinking Duration (s)", - "Extrusion Pressure (kPa)", - "Nozzle Movement Speed (mm/s)", - "Nozzle Diameter (Β΅m)", - "Syringe Temperature (Β°C)", - "Substrate Temperature (Β°C)", -] - -@st.cache_resource -def load_prediction_preprocessors(): - return { - "preprocessor": joblib.load(PREPROCESSOR_DIR / "preprocessor.pkl"), - "feature_cols": joblib.load(PREPROCESSOR_DIR / "feature_cols.pkl"), - "printability_encoder": joblib.load(PREPROCESSOR_DIR / MODEL_TASKS["printability"]["label_encoder"]), - "cell_response_encoder": joblib.load(PREPROCESSOR_DIR / MODEL_TASKS["cell_response"]["label_encoder"]), - } - -def model_display_name(path, prefix): - name = path.stem.replace("_model", "") - if name.startswith(prefix): - name = name[len(prefix):] - return f"{name} ({path.suffix.lstrip('.')})" - -def model_rank_key(display_name, task_key): - base_name = display_name.rsplit(" (", 1)[0] - rank_aliases = { - "1D_CNN": "1D CNN", - "FT_Transformer": "FT Transformer", - "NODE_Lite": "NODE Lite", - "TabNet_Lite": "TabNet Lite", - "TabPFN_2.6": "TabPFN 2.6", - "TabICL_v2": "TabICL v2", - "ExtraTree": "ExtraTreeClassifier", - } - rank_name = rank_aliases.get(base_name, base_name) - if base_name == "MLP" and display_name.endswith("(pth)"): - rank_name = "MLP DL" - ranking = MODEL_RANKINGS[task_key] - rank = ranking.index(rank_name) if rank_name in ranking else len(ranking) - return rank, base_name.lower(), display_name - -def discover_model_options(task_key): - task = MODEL_TASKS[task_key] - files = [] - for suffix in ("*.pkl", "*.joblib", "*.pth"): - files.extend(task["folder"].glob(suffix)) - options = {model_display_name(path, task["prefix"]): str(path) for path in files} - return dict(sorted(options.items(), key=lambda item: model_rank_key(item[0], task_key))) - -def parse_architecture(path, task_key): - stem = Path(path).stem.replace("_model", "") - prefix = MODEL_TASKS[task_key]["prefix"] - return stem[len(prefix):] if stem.startswith(prefix) else stem - -def build_torch_model(architecture, input_dim, out_dim, cfg): - import torch - import torch.nn as nn - - activation_funcs = { - "relu": nn.ReLU, "tanh": nn.Tanh, "GELU": nn.GELU, - "SELU": nn.SELU, "ELU": nn.ELU, "SiLU": nn.SiLU, - } - act = activation_funcs[cfg["activation_func"]] - - class ResidualBlock(nn.Module): - def __init__(self): - super().__init__() - self.linear = nn.Linear(cfg["hidden_dim"], cfg["hidden_dim"]) - self.bn = nn.BatchNorm1d(cfg["hidden_dim"]) - self.act = act() - self.dropout = nn.Dropout(cfg["dropout"]) - def forward(self, x): - return x + self.dropout(self.act(self.bn(self.linear(x)))) - - class TissueResNet(nn.Module): - def __init__(self): - super().__init__() - self.input_layer = nn.Sequential( - nn.Linear(input_dim, cfg["hidden_dim"]), - nn.BatchNorm1d(cfg["hidden_dim"]), - act(), - ) - self.blocks = nn.ModuleList([ResidualBlock() for _ in range(cfg["n_layers"])]) - self.output_layer = nn.Linear(cfg["hidden_dim"], out_dim) - def forward(self, x): - x = self.input_layer(x) - for block in self.blocks: - x = block(x) - return self.output_layer(x) - - class StandardMLP(nn.Module): - def __init__(self): - super().__init__() - layers = [] - in_dim = input_dim - for _ in range(cfg["n_layers"]): - layers.extend([ - nn.Linear(in_dim, cfg["hidden_dim"]), - nn.BatchNorm1d(cfg["hidden_dim"]), - act(), - nn.Dropout(cfg["dropout"]), - ]) - in_dim = cfg["hidden_dim"] - layers.append(nn.Linear(cfg["hidden_dim"], out_dim)) - self.network = nn.Sequential(*layers) - def forward(self, x): - return self.network(x) - - class Tabular1DCNN(nn.Module): - def __init__(self): - super().__init__() - layers = [] - in_channels = 1 - for _ in range(cfg["n_layers"]): - layers.extend([ - nn.Conv1d(in_channels, cfg["hidden_dim"], kernel_size=3, padding=1), - nn.BatchNorm1d(cfg["hidden_dim"]), - act(), - nn.Dropout(cfg["dropout"]), - ]) - in_channels = cfg["hidden_dim"] - self.conv_net = nn.Sequential(*layers) - self.pool = nn.AdaptiveAvgPool1d(1) - self.fc = nn.Linear(cfg["hidden_dim"], out_dim) - def forward(self, x): - x = self.conv_net(x.unsqueeze(1)) - return self.fc(self.pool(x).squeeze(2)) - - class FTTransformer(nn.Module): - def __init__(self): - super().__init__() - self.d_token = max(4, (cfg["hidden_dim"] // 4) * 4) - self.feature_embeddings = nn.ModuleList([nn.Linear(1, self.d_token) for _ in range(input_dim)]) - self.cls_token = nn.Parameter(torch.randn(1, 1, self.d_token)) - encoder_layer = nn.TransformerEncoderLayer( - d_model=self.d_token, nhead=4, dropout=cfg["dropout"], batch_first=True - ) - self.transformer = nn.TransformerEncoder(encoder_layer, num_layers=cfg["n_layers"]) - self.fc = nn.Linear(self.d_token, out_dim) - def forward(self, x): - batch_size = x.size(0) - tokens = [self.feature_embeddings[i](x[:, i:i+1]).unsqueeze(1) for i in range(x.size(1))] - x_emb = torch.cat([self.cls_token.expand(batch_size, -1, -1)] + tokens, dim=1) - return self.fc(self.transformer(x_emb)[:, 0, :]) - - class TabNetLite(nn.Module): - def __init__(self): - super().__init__() - self.n_steps = max(1, cfg["n_layers"]) - self.initial_bn = nn.BatchNorm1d(input_dim) - self.transformers = nn.ModuleList([ - nn.Sequential( - nn.Linear(input_dim, cfg["hidden_dim"]), - nn.BatchNorm1d(cfg["hidden_dim"]), - act(), - nn.Dropout(cfg["dropout"]), - ) - for _ in range(self.n_steps) - ]) - self.attentions = nn.ModuleList([ - nn.Sequential( - nn.Linear(cfg["hidden_dim"], input_dim), - nn.BatchNorm1d(input_dim), - nn.Softmax(dim=-1), - ) - for _ in range(self.n_steps) - ]) - self.fc_out = nn.Linear(cfg["hidden_dim"], out_dim) - def forward(self, x): - x = self.initial_bn(x) - out_agg, prior = 0, torch.ones_like(x) - feat_rep = self.transformers[0](x) - for step in range(self.n_steps): - mask = self.attentions[step](feat_rep) * prior - prior = prior * (1.0 - mask) - feat_rep = self.transformers[step](x * mask) - out_agg += feat_rep - return self.fc_out(out_agg) - - class NeuralDecisionForest(nn.Module): - def __init__(self): - super().__init__() - self.n_trees = max(1, cfg["hidden_dim"] // 16) - self.depth = max(2, cfg["n_layers"] + 1) - self.n_leaves = 2 ** self.depth - self.trees = nn.ModuleList([ - nn.Sequential( - nn.Linear(input_dim, self.n_leaves), - nn.Dropout(cfg["dropout"]), - nn.Softmax(dim=-1), - ) - for _ in range(self.n_trees) - ]) - self.leaf_weights = nn.Parameter(torch.randn(self.n_trees, self.n_leaves, out_dim)) - def forward(self, x): - out = 0 - for i, tree in enumerate(self.trees): - out += torch.matmul(tree(x), self.leaf_weights[i]) - return out / self.n_trees - - builders = { - "ResNet": TissueResNet, - "MLP": StandardMLP, - "1D_CNN": Tabular1DCNN, - "FT_Transformer": FTTransformer, - "TabNet_Lite": TabNetLite, - "NODE_Lite": NeuralDecisionForest, - } - return builders[architecture]() - -@st.cache_resource -def load_prediction_model(path, task_key, input_dim, out_dim): - path = Path(path) - if path.suffix in {".pkl", ".joblib"}: - return {"kind": "sklearn", "model": joblib.load(path)} - - if path.suffix == ".pth": - import torch - architecture = parse_architecture(path, task_key) - cfg = DL_MODEL_CONFIGS.get((task_key, architecture)) - if cfg is None: - raise ValueError(f"No architecture configuration found for {path.name}.") - model = build_torch_model(architecture, input_dim, out_dim, cfg) - state_dict = torch.load(path, map_location="cpu") - model.load_state_dict(state_dict) - model.eval() - return {"kind": "torch", "model": model} - - raise ValueError(f"Unsupported model file: {path.name}") - -def softmax(values): - values = np.asarray(values, dtype=float) - values = values - np.max(values) - exp_values = np.exp(values) - return exp_values / exp_values.sum() - -def expected_class_value(model_bundle, x_raw, preprocessor, label_encoder): - x_model = preprocessor.transform(x_raw).astype(np.float32) - - if model_bundle["kind"] == "torch": - import torch - with torch.no_grad(): - logits = model_bundle["model"](torch.tensor(x_model, dtype=torch.float32)).numpy()[0] - probs = softmax(logits) - labels = label_encoder.classes_.astype(float) - return float(np.dot(probs, labels)) - - model = model_bundle["model"] - if hasattr(model, "predict_proba"): - probs = model.predict_proba(x_model)[0] - classes = np.asarray(model.classes_, dtype=int) - labels = label_encoder.inverse_transform(classes).astype(float) - return float(np.dot(probs, labels)) - - if hasattr(model, "decision_function"): - scores = np.asarray(model.decision_function(x_model)[0]) - if scores.ndim == 0: - p_high = 1.0 / (1.0 + np.exp(-scores)) - probs = np.array([1.0 - p_high, p_high]) - else: - probs = softmax(scores) - classes = np.asarray(model.classes_, dtype=int) - labels = label_encoder.inverse_transform(classes).astype(float) - return float(np.dot(probs, labels)) - - pred = np.asarray(model.predict(x_model), dtype=int) - return float(label_encoder.inverse_transform(pred)[0]) - -prediction_assets = load_prediction_preprocessors() -preprocessor = prediction_assets["preprocessor"] -feature_cols = prediction_assets["feature_cols"] -label_encoder_print = prediction_assets["printability_encoder"] -label_encoder_cell = prediction_assets["cell_response_encoder"] - -sample_for_shape = {col: 0.0 for col in feature_cols} -sample_for_shape["Cell Line"] = CELL_LINE_OPTIONS[0] -preprocessed_input_dim = preprocessor.transform(pd.DataFrame([sample_for_shape])[feature_cols]).shape[1] - -@st.dialog("Optimization Trials") -def show_trial_guidance(): - st.markdown( - """ - The trial count controls how many candidate scaffold settings Optuna tests before choosing the best WSSQ. - - **Recommended range:** 100-1000 trials. - - **Default:** 300 trials, which is a balanced choice for normal use. - - **Runtime impact:** running time grows roughly in proportion to the number of trials. Use 50-100 for a quick test, 300 for a balanced run, and 500-1000 when you want a more thorough search and can wait longer. - """ - ) - -@st.dialog("Weighted Synergistic Scaffold Quality (WSSQ)", width="large") -def show_wssq_guidance(): - st.markdown( - """ - WSSQ is the optimization score used in MLATE to combine **printability** and **cell response** into one scaffold-quality objective. - - WSSQ was introduced to handle two practical needs: acellular 3D-printed scaffolds, where cell response is not applicable in the same way, and bioprinted scaffolds, where biological response is central to scaffold quality. - - The app normalizes printability and cell response, then combines them using two components: - - - **Weighted harmonic mean:** rewards balanced high values and penalizes weak performance in either target. - - **Weighted multiplicative component:** captures synergy between printability and cell response. - - The final WSSQ is scaled from 0 to 100%. If printability is 0, WSSQ is 0. If cell response is at the minimum biological-response level, the score falls back to normalized printability. Otherwise, both targets contribute according to the sidebar weights. - - Practically, a scaffold with excellent cell response but poor printability can score lower than a scaffold with slightly lower cell response but better balance, because WSSQ is designed to favor experimentally useful, well-balanced scaffold candidates. - """ - ) - -@st.dialog("Model Selection Guide", width="large") -def show_model_selection_guidance(): - st.markdown( - """ - Models are ranked by F1 score and accuracy. The highest-ranked available model appears first in each dropdown. - - Use the top-ranked models when you want the strongest benchmarked predictive performance. If a selected model fails to load because of local package-version incompatibility, choose the next ranked model in the same task until the environment is aligned with the model artifacts. - """ - ) - tab_print, tab_cell = st.tabs(["Printability", "Cell Response"]) - with tab_print: - st.markdown(PERFORMANCE_GUIDE["printability"]) - with st.expander("Show more"): - st.markdown(PERFORMANCE_GUIDE_FULL["printability"]) - with tab_cell: - st.markdown(PERFORMANCE_GUIDE["cell_response"]) - with st.expander("Show more"): - st.markdown(PERFORMANCE_GUIDE_FULL["cell_response"]) - -@st.dialog("How to Get a Gemini API Key") -def show_gemini_api_key_guidance(): - st.markdown( - """ - To generate a fabrication procedure, you need a Gemini API key from Google. Creating a key only takes a minute, and Google provides a free tier. - - 1. Open [Google AI Studio API Keys](https://aistudio.google.com/app/apikey) and sign in with your Google/Gmail account if prompted. - 2. If this is your first visit, accept the terms of service and continue. - 3. Click **Get API key** or **Create API key**. - 4. Choose an existing Google Cloud project, or select **Create API key in new project**. - 5. Copy the generated key, return to this app, and paste it into the **Gemini API Key** box. - - **Important:** Treat your API key like a password. Do not share it publicly or paste it into files that will be uploaded online. This app uses your key only for the current protocol generation request and does not save it. - """ - ) - -if 'bio_rows' not in st.session_state: - st.session_state.bio_rows = [{ - 'mat': BIOMATERIAL_OPTIONS[0], - 'min': 0.0, 'max': 10.0, 'step': 0.1 - }] - -if 'density_range' not in st.session_state: - st.session_state.density_range = {'min': 0.0, 'max': 10.0, 'step': 0.1} - -if 'pp_ranges' not in st.session_state: - st.session_state.pp_ranges = { - "Physical Crosslinking Duration (s)": {'min': 0.0, 'max': 300.0, 'step': 5.0}, - "Photo Crosslinking Duration (s)": {'min': 0.0, 'max': 180.0, 'step': 5.0}, - "Extrusion Pressure (kPa)": {'min': 5.0, 'max': 200.0, 'step': 5.0}, - "Nozzle Movement Speed (mm/s)": {'min': 1.0, 'max': 20.0, 'step': 0.5}, - "Nozzle Diameter (Β΅m)": {'min': 100.0,'max': 1000.0, 'step': 50.0}, - "Syringe Temperature (Β°C)": {'min': 4.0, 'max': 40.0, 'step': 1.0}, - "Substrate Temperature (Β°C)": {'min': 4.0, 'max': 37.0, 'step': 1.0}, - } - -# --- Sidebar UI for Weights --- -st.sidebar.header("Optimization Weights") -# User only controls Cell Response (0 to 100) -w_cell_pct = st.sidebar.slider("Cell Response Weight (%)", min_value=0, max_value=100, value=70, step=5) -# Printability is dynamically calculated and cannot be changed manually -w_print_pct = 100 - w_cell_pct -st.sidebar.number_input("Printability Weight (%)", value=w_print_pct, disabled=True, help="Auto-calculated to ensure sum is 100%") -if st.sidebar.button("What is WSSQ?", use_container_width=True): - show_wssq_guidance() - -# Convert back to 0.0 - 1.0 for the mathematical formula -w_cell = w_cell_pct / 100.0 -w_print = w_print_pct / 100.0 - -print_model_options = discover_model_options("printability") -cell_model_options = discover_model_options("cell_response") - -if not print_model_options or not cell_model_options: - st.error("No selectable prediction models were found in the models folder.") - st.stop() - -selected_print_model_name = st.sidebar.selectbox( - "Printability Model", - list(print_model_options.keys()), - key="printability_model_select", -) -selected_cell_model_name = st.sidebar.selectbox( - "Cell Response Model", - list(cell_model_options.keys()), - key="cell_response_model_select", -) -if st.sidebar.button("Model Selection Guide", use_container_width=True): - show_model_selection_guidance() - -n_trials = st.sidebar.number_input( - "Optimization Trials", - min_value=10, - max_value=10000, - value=300, - step=50, - help="Number of Optuna trials used when you click Optimize WSSQ.", -) -if st.sidebar.button("Trial Count Help", use_container_width=True): - show_trial_guidance() - -gemini_key = st.sidebar.text_input( - "Gemini API Key", - value=os.getenv("GEMINI_API_KEY", ""), - type="password", - help="Used only when generating the LLM-based fabrication procedure.", -) -if st.sidebar.button("How to Get API Key", use_container_width=True): - show_gemini_api_key_guidance() - -gemini_model = st.sidebar.selectbox( - "Gemini Model", - GEMINI_MODELS, - index=0, - key="gemini_model_select", -) - -st.title("MLATE: Machine Learning Applications in Tissue Engineering") -st.markdown( - "

" - "A Data-driven Cross-tissue Machine Learning Framework for Inverse design of 3D (Bio)printing Scaffolds " - "For more details, please refer to and cite our paper: " - "https://doi.org/xxx" - "

", - unsafe_allow_html=True -) - -st.subheader("Biomaterials (enter range for each)") -if st.button("βž• Add Biomaterial"): - used = {r['mat'] for r in st.session_state.bio_rows} - available = [m for m in BIOMATERIAL_OPTIONS if m not in used] - if available: - st.session_state.bio_rows.append({ - 'mat': available[0], 'min': 0.0, 'max': 10.0, 'step': 0.1 - }) - st.rerun() - -for i, row in enumerate(st.session_state.bio_rows): - used_except_current = { - r['mat'] for idx, r in enumerate(st.session_state.bio_rows) if idx != i - } - options = [m for m in BIOMATERIAL_OPTIONS if m not in used_except_current] - - c1, c2, c3, c4, c5 = st.columns([2, 1, 1, 1, 0.3]) - mat = c1.selectbox( - "Biomaterial", options, - index=options.index(row['mat']) if row['mat'] in options else 0, - key=f"bio_mat_{i}", - label_visibility="collapsed", - ) - st.session_state.bio_rows[i]['mat'] = mat - - mn = c2.number_input( - "Min", min_value=0.0, max_value=row['max'], - value=row['min'], step=row['step'], key=f"bio_min_{i}" - ) - mx = c3.number_input( - "Max", min_value=row['step'], max_value=100.0, - value=max(row['max'], row['step']), step=row['step'], - key=f"bio_max_{i}" - ) - st.session_state.bio_rows[i].update(min=mn, max=mx) - - st.session_state.bio_rows[i]['step'] = c4.number_input( - "Step", min_value=0.0, - max_value=(mx - mn) if mx > mn else 0.1, - value=row['step'], step=0.1, key=f"bio_step_{i}" - ) - - if c5.button("❌", key=f"rem_{i}"): - st.session_state.bio_rows.pop(i) - st.rerun() - -st.markdown("---") - -st.subheader("Cell Line & Density (10^6 cells/ml)") -col1, col2, col3, col4 = st.columns([2,1,1,1]) - -cell_line = col1.selectbox("Cell Line", CELL_LINE_OPTIONS, key="cell_line_select") - -if cell_line == "NoCellCultured": - st.info("πŸ§ͺ **Acellular 3D printing mode** – No cells will be included. Cell density is forced to 0.") - st.session_state.density_range.update({'min': 0.0, 'max': 0.0, 'step': 0.0}) - - col2.number_input("Min Density", value=0.0, disabled=True, key="cd_min") - col3.number_input("Max Density", value=0.0, disabled=True, key="cd_max") - col4.number_input("Step", value=0.0, disabled=True, key="cd_step") -else: - if st.session_state.density_range.get('max', 0) <= 0.1: - st.session_state.density_range.update({'min': 1.0, 'max': 20.0, 'step': 0.5}) - - dr = st.session_state.density_range - dmin = col2.number_input( - "Min Density", - min_value=0.0, - max_value=dr['max'], - value=dr['min'], - step=dr['step'], - key="cd_min" - ) - dmax = col3.number_input( - "Max Density", - min_value=dr['step'], - max_value=1000.0, - value=max(dr['max'], dr['step']), - step=dr['step'], - key="cd_max" - ) - dstep = col4.number_input( - "Step", - min_value=0.0, - max_value=(dmax - dmin) if dmax > dmin else 0.1, - value=dr['step'], - step=0.1, - key="cd_step" - ) - st.session_state.density_range.update({'min': dmin, 'max': dmax, 'step': dstep}) - -st.markdown("---") - -st.subheader("Crosslinking Settings") - -col_cross1, col_cross2 = st.columns(2) - -disable_physical = col_cross1.checkbox( - "Disable Physical Crosslinking", - value=False, - help="Check if you do not want physical/ionic crosslinking (e.g. CaClβ‚‚ bath, temperature-induced)" -) - -disable_photo = col_cross2.checkbox( - "Disable Photo Crosslinking", - value=False, - help="Check if you do not want UV/visible light crosslinking" -) - -st.subheader("Printing Parameters (enter range)") - -for name in PRINT_PARAM_NAMES: - if name == "Physical Crosslinking Duration (s)" and disable_physical: - st.session_state.pp_ranges[name].update({'min': 0.0, 'max': 0.0, 'step': 0.0}) - c1, c2, c3, c4 = st.columns([2,1,1,1]) - c1.write(name + " (DISABLED)") - c2.number_input("Min", value=0.0, disabled=True, key=f"pp_min_{name}") - c3.number_input("Max", value=0.0, disabled=True, key=f"pp_max_{name}") - c4.number_input("Step", value=0.0, disabled=True, key=f"pp_step_{name}") - continue - - elif name == "Photo Crosslinking Duration (s)" and disable_photo: - st.session_state.pp_ranges[name].update({'min': 0.0, 'max': 0.0, 'step': 0.0}) - c1, c2, c3, c4 = st.columns([2,1,1,1]) - c1.write(name + " (DISABLED)") - c2.number_input("Min", value=0.0, disabled=True, key=f"pp_min_{name}") - c3.number_input("Max", value=0.0, disabled=True, key=f"pp_max_{name}") - c4.number_input("Step", value=0.0, disabled=True, key=f"pp_step_{name}") - continue - - pmin = st.session_state.pp_ranges[name]['min'] - pmax = st.session_state.pp_ranges[name]['max'] - pstep = st.session_state.pp_ranges[name]['step'] - - c1, c2, c3, c4 = st.columns([2,1,1,1]) - c1.write(name) - - pmin = c2.number_input( - "Min", min_value=0.0, max_value=pmax, - value=pmin, step=pstep, key=f"pp_min_{name}" - ) - pmax = c3.number_input( - "Max", min_value=pstep, max_value=10000.0, - value=max(pmax, pstep), step=pstep, key=f"pp_max_{name}" - ) - pstep = c4.number_input( - "Step", min_value=0.0, - max_value=(pmax - pmin) if pmax > pmin else 1.0, - value=pstep, step=max(1e-3, pstep/10), - key=f"pp_step_{name}" - ) - st.session_state.pp_ranges[name].update(min=pmin, max=pmax, step=pstep) -st.markdown("---") - -if st.button("Optimize WSSQ"): - with st.spinner("Running Optuna…"): - try: - model_print = load_prediction_model( - print_model_options[selected_print_model_name], - "printability", - preprocessed_input_dim, - len(label_encoder_print.classes_), - ) - model_cell = load_prediction_model( - cell_model_options[selected_cell_model_name], - "cell_response", - preprocessed_input_dim, - len(label_encoder_cell.classes_), - ) - except Exception as exc: - st.error( - "Could not load the selected prediction model. This model will only work when the " - "running environment matches the saved artifact dependencies. For HistGradientBoosting, " - "use numpy>=2.0 in the Python environment that runs Streamlit.\n\n" - f"Details:\n{exc}" - ) - st.stop() - - def objective(trial): - bi_vals = { - r['mat']: trial.suggest_float( - f"bio__{r['mat']}", r['min'], r['max'], step=r['step'] - ) - for r in st.session_state.bio_rows - } - for m in BIOMATERIAL_OPTIONS: - bi_vals.setdefault(m, 0.0) - - cd = 0.0 if cell_line=="NoCellCultured" else trial.suggest_float( - "cell_density", dr['min'], dr['max'], step=dr['step'] - ) - - pp_vals = { - name: trial.suggest_float( - f"pp__{name}", - st.session_state.pp_ranges[name]['min'], - st.session_state.pp_ranges[name]['max'], - step=st.session_state.pp_ranges[name]['step'] - ) - for name in PRINT_PARAM_NAMES - } - - feat = {**bi_vals, **pp_vals} - feat["Cell Density (cells/mL)"] = cd - feat["Cell Line"] = cell_line - - X = pd.DataFrame([feat]).reindex(columns=feature_cols, fill_value=0.0) - - exp_p = expected_class_value(model_print, X, preprocessor, label_encoder_print) - exp_c = expected_class_value(model_cell, X, preprocessor, label_encoder_cell) - - np.random.seed(42) - # Use dynamic weights from the sidebar sliders - return scaffold_quality_combined( - exp_p, - exp_c, - weight_printability=w_print, - weight_cell_response=w_cell - ) - - sampler = optuna.samplers.TPESampler( - seed=42, - n_startup_trials=30, - multivariate=True, - group=True, - consider_prior=True - ) - - study = optuna.create_study( - direction="maximize", - sampler=sampler, - pruner=optuna.pruners.MedianPruner() - ) - study.optimize(objective, n_trials=int(n_trials)) - - # Store results in session state to persist after rerun - st.session_state.best_params = study.best_trial.params - st.session_state.best_value = study.best_trial.value - st.session_state.optimized_cell_line = cell_line - -if 'best_params' in st.session_state: - st.success(f"Best WSSQ: **{st.session_state.best_value:.3f}**") - best_df = pd.Series(st.session_state.best_params, name="value") \ - .rename_axis("parameter") \ - .to_frame() - st.table(best_df) - - st.markdown("---") - st.subheader("Customize Fabrication Protocol") - user_inquiry = st.text_area( - "Add specific limitations, equipment, or extra requirements:", - placeholder="e.g., I only have a 25G nozzle available, or I need to use a specific UV intensity of 10mW/cmΒ²...", - key="user_inquiry" - ) - - if st.button("Generate Fabrication Procedure"): - if not gemini_key: - st.error("Please enter your Gemini API key in the sidebar before generating a fabrication procedure.") - st.stop() - - with st.spinner("Generating rigorous fabrication procedure…"): - client = genai.Client(api_key=gemini_key) - - formatted_params = "\n".join([ - f"- {k.replace('bio__', 'Biomaterial: ').replace('pp__', 'Print Setting: ')}: {v:.2f}" - for k, v in st.session_state.best_params.items() - ]) - - # Base prompt (remains unchanged) - base_prompt = ( - f"Please act as a senior tissue engineer with 15+ years of hands-on experience in 3D bioprinting for regenerative medicine. " - f"Write a **highly practical, bench-ready laboratory fabrication protocol** for fabricating a scaffold. " - f"Assume the reader is an experienced experimentalist who routinely works in a tissue engineering lab.\n\n" - f"**Use exactly these inputs to tailor every step:**\n" - f"Target Cell Line: {st.session_state.optimized_cell_line}\n" - f"Parameters:\n{formatted_params}\n\n" - f"**Critical requirements for the protocol (you MUST follow all of them):**\n" - f"β€’ If Target Cell Line is 'NoCellCultured', this is **acellular 3D printing** (not bioprinting). Remove all references to cells, cell viability, cell density, and cell culturing. The final scaffold is cell-free. Change section 6 title to 'Post-Printing Incubation & Storage Instructions' and adapt its content accordingly.\n" - f"β€’ If any suggested parameter is physically unrealistic (e.g. nozzle diameter 9 Β΅m or syringe temp 2Β°C) or the nozzle diameter is very small relative to the cell diameter of the target cell line (when cells are used), adjust it slightly in the protocol and explicitly note the adjustment with justification.\n" - f"β€’ Every quantity must be given in precise, measurable lab units (e.g., 2.5 mL, 1.2 % w/v, 10 mg/mL, 37 Β°C, 5 min, 150 rpm).\n" - f"β€’ Include exact timings, temperatures, and workflow order to protect structural fidelity (and cell viability >85 % post-print when cells are used).\n" - f"β€’ Anticipate and explicitly address common bioprinting pitfalls relevant to the given parameters (nozzle clogging, shear-induced cell death, premature gelation, filament fusion, air bubbles, etc.) and give precise mitigation steps.\n" - f"β€’ Use only reagents and equipment that are standard in tissue engineering labs; if a specific brand/model is implied by the parameters, note a common equivalent.\n" - f"β€’ Include simple quality-control checkpoints (visual inspection, live/dead staining timing when cells are used, etc.).\n\n" - f"Your response must be structured **exactly** with the following sections (no extra sections, no introductory text, no summary, no conclusions):\n" - f"1. Required Materials & Equipment\n" - f"2. Sterilization & Safety Precautions\n" - f"3. Bioink Preparation\n" - f"4. 3D Bioprinting Settings & Execution\n" - f"5. Post-processing & Crosslinking\n" - f"6. Cell Culturing & Incubation Instructions\n" - ) - - # Append user inquiry if provided - final_prompt = base_prompt - if user_inquiry: - final_prompt += f"\n**Additional User Constraints & Inquiries (Integrate these into the protocol):**\n{user_inquiry}" - - resp = client.models.generate_content( - model=gemini_model, - contents=final_prompt, - config=types.GenerateContentConfig( - system_instruction=( - "You are a senior tissue engineer and expert experimentalist specializing in translating optimized bioprinting parameters into reproducible, high-viability laboratory protocols. " - "Your protocols are used daily by PhD students and post-docs in regenerative medicine labs. " - "You always prioritize: (1) maximum cell viability and function, (2) structural fidelity of the printed construct, (3) workflow efficiency under sterile conditions, and (4) safety. " - "Write in clear, imperative, step-by-step language with numbered or bulleted sub-steps. " - "Never be vague β€” give exact volumes, times, temperatures, speeds, and concentrations. " - "Never add disclaimers or theoretical background unless explicitly asked." - ), - temperature=0.1, - top_p=0.85, - max_output_tokens=6144 - ) - ) - - st.markdown("## Fabrication Procedure") - st.markdown(resp.text) +""" +MLATE V3 β€” scaffold optimisation and protocol generation +======================================================== + + streamlit run 06_webapp/app.py + +A thin interface over `mlate.optimize`, `mlate.wssq` and `mlate.protocol`. The +previous release put the objective function, the WSSQ formula and the language +model call inside this file, which meant none of them could be run, tested or +reported without launching a browser. Everything scientific now lives in the +package; this file collects inputs, calls it, and displays the result. + +Deployable to a Hugging Face Space: model artefacts are located relative to +this file, and the only corpus data required at run time is the trimmed +reference table that `build_app_data.py` writes beside the app. +""" + +from __future__ import annotations + +import json +import sys +from pathlib import Path +from types import SimpleNamespace + +import joblib +import numpy as np +import pandas as pd +import streamlit as st + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE.parent)) +sys.path.insert(0, str(HERE)) + +from mlate import config as cfg # noqa: E402 +from mlate import optimize as opt # noqa: E402 +from mlate import protocol as proto # noqa: E402 +from mlate import serving # noqa: E402 +from mlate import wssq as wssq_mod # noqa: E402 + +from biomaterials import BIOMATERIAL_OPTIONS, BIOMATERIAL_RANGES # noqa: E402 +from cell_lines import (ACELLULAR_TOKEN, CELL_DENSITY_RANGES, # noqa: E402 + CELL_LINE_COUNTS, CELL_LINE_OPTIONS) +from model_performance import (PERFORMANCE_GUIDE, # noqa: E402 + PERFORMANCE_GUIDE_FULL) + +st.set_page_config(page_title="MLATE V3", page_icon="🧬", layout="wide") + +# Deployment layout: a Space carries deploy/models next to the app, a checkout +# has it at the repository root. Both are tried so the same file runs in either. +MODEL_ROOTS = [HERE / "deploy" / "models", cfg.ROOT / "deploy" / "models"] +PRINT_PARAMS = ["Physical Crosslinking Duration (s)", + "Photo Crosslinking Duration (s)", + "Extrusion Pressure (kPa)", + "Nozzle Movement Speed (mm/s)", + "Nozzle Diameter (Β΅m)", + "Syringe Temperature (Β°C)", + "Substrate Temperature (Β°C)"] +CROSSLINK = PRINT_PARAMS[:2] +PROTOCOL_PARAMS = PRINT_PARAMS[2:] + + +def models_root() -> Path | None: + for r in MODEL_ROOTS: + if (r / "classifiers").exists(): + return r + return None + + +# ── loading ────────────────────────────────────────────────────────────────── + +@st.cache_resource(show_spinner=False) +def load_preprocessor(): + """The release preprocessor, fitted on every row for inference.""" + root = models_root() + d = root / "preprocessors" + return (joblib.load(d / "preprocessor.pkl"), + joblib.load(d / "input_columns.pkl")) + + +@st.cache_resource(show_spinner=False) +def load_manifest() -> dict: + root = models_root() + p = root / "deployment_manifest.json" + return json.loads(p.read_text(encoding="utf-8")) if p.exists() else {} + + +@st.cache_data(show_spinner=False) +def available_models(task: str) -> list[dict]: + """ + Every model offered for one target, ranked by benchmarked weighted F1. + + All three families, not only the pickled classifiers: the previous release + listed the conventional models alone, which meant TabICL - the strongest + printability model on the benchmark - was exported, reported and then not + offered. Discovery costs a filename parse; an estimator is opened when it + is chosen. + + Random-split artefacts only. The application serves prediction inside the + design space the corpus covers - adjusting a concentration, swapping a cell + line, moving a pressure within observed ranges - which is the interpolation + regime the random protocol estimates. The study-grouped models are the + conservative bound for an unseen laboratory and are reported in the + manuscript, but they are tuned for a harder task than the one performed + here. The foundation models are split-independent and appear under both. + """ + root = models_root() + if root is None: + return [] + return [{"name": st_.name, "family": st_.family, "path": str(st_.path), + "weighted_f1": st_.weighted_f1, "cost": st_.cost_hint} + for st_ in serving.discover(root, task, "random")] + + +def model_label(entry: dict) -> str: + """Menu text: the model, what kind it is, and what it scored.""" + kind = {"ml": "", "dl": " Β· deep", "foundation": " Β· foundation"} + return (f"{entry['name']}{kind.get(entry['family'], '')} Β· " + f"F1 {entry['weighted_f1']:.3f}") + + +@st.cache_resource(show_spinner=False) +def load_model(path_str: str, family: str): + """ + Open one artefact. Cached, because a foundation model re-supplies its + 2,646-row context on first use and there is no reason to pay that twice. + """ + return serving.load(Path(path_str), family) + + +@st.cache_data(show_spinner=False) +def load_corpus() -> tuple[pd.DataFrame | None, object]: + """ + The observed formulations, shipped beside the app by `build_app_data.py`. + + Needed for the two checks that a prediction alone cannot make: how far the + proposed formulation sits from anything published, and which real + formulations are closest to it. Returns (None, None) if the file is absent + rather than failing, so the application still optimises; the interface then + says the check could not be performed instead of implying it passed. + """ + table, meta = HERE / "corpus_reference.parquet", HERE / "corpus_reference.json" + if not (table.exists() and meta.exists()): + return None, None + groups = json.loads(meta.read_text(encoding="utf-8")) + return pd.read_parquet(table), SimpleNamespace(**groups) + + +@st.cache_data(show_spinner=False, ttl=3600) +def list_llms() -> tuple[list[tuple], bool]: + """ + The language models on offer, checked against OpenRouter's live catalogue. + + Checked rather than trusted because the catalogue turns over quickly: five + of the eleven identifiers shipped with the previous revision had been + withdrawn within weeks, and a withdrawn identifier fails only when the user + presses Generate, after an optimisation has already been paid for. Cached + for an hour so the check costs one request per session. + + No key is passed. The catalogue is public, and Streamlit's cache is shared + across every session of a deployment, so a secret used as a cache key would + be a secret held on behalf of all of them. + """ + return proto.available_models(None) + + +# ── help dialogs ───────────────────────────────────────────────────────────── + +@st.dialog("Weighted Synergistic Scaffold Quality (WSSQ)", width="large") +def show_wssq_guidance(): + st.markdown( + """ +WSSQ combines **printability** and **cell response** into a single score the +optimiser can maximise. + +Both components are combined by two *conjunctive* means β€” a weighted harmonic +mean and a weighted geometric mean, averaged. Conjunctive means collapse toward +zero if either component does, so neither objective can be traded away: a +scaffold that prints perfectly but supports no cells does not score well. An +ordinary average would allow exactly that. + +Two boundary rules apply. A formulation that does not extrude (printability 0) +scores 0. An **acellular** formulation has no cell response to assess, so it is +scored on printability alone rather than being penalised for a biological +outcome that does not apply to it. + +The **cell-response weight** below is yours to set. Cell response carries more +than biology: pore size, porosity, interconnectivity and stiffness are not known +before fabrication and cannot be model inputs, but they strongly shape how cells +behave, so cell response acts as a proxy for them. + """) + + +@st.dialog("Choosing a model", width="large") +def show_model_guidance(): + st.markdown( + "Ranked by weighted F1 on the held-out test partition of the " + "random split (n = 530 records per target).\n\n" + "All three families are offered: the conventional classifiers, the six " + "deep networks, and the three in-context foundation models. The " + "highest-ranked model appears first in each menu.\n\n" + "**What the choice costs.** A foundation model carries the corpus " + "rather than fitted parameters and re-reads all 2,646 records on every " + "pass, so it is slower than a conventional classifier - measured at " + "about 1 s per batch of candidates on a GPU and about 17 s without " + "one, against 80 ms. Candidates are scored in batches of 32 precisely " + "so that this is paid once per batch rather than once per candidate; a " + "150-trial search is a few seconds with a conventional model, under a " + "minute with a foundation model on a GPU, and around three minutes on " + "CPU.\n\n" + "Models are refitted on the complete dataset for deployment. If one " + "fails to load because of a local package-version mismatch, choose the " + "next in the list.") + tp, tc = st.tabs(["Printability", "Cell Response"]) + with tp: + st.markdown(PERFORMANCE_GUIDE["printability"]) + with st.expander("All models"): + st.markdown(PERFORMANCE_GUIDE_FULL["printability"]) + with tc: + st.markdown(PERFORMANCE_GUIDE["cell_response"]) + with st.expander("All models"): + st.markdown(PERFORMANCE_GUIDE_FULL["cell_response"]) + + +@st.dialog("Optimisation trials") +def show_trial_guidance(): + st.markdown( + "Each trial is one candidate formulation, scored by the two models " + "and combined into WSSQ. The search is Bayesian: it models which " + "regions of the space produce good scores and samples there, so later " + "trials are better targeted than earlier ones.\n\n" + "The first 30 trials or so explore broadly and their scores mean " + "little on their own. **100 to 300 trials** is a reasonable range; " + "more trials help most when many variables are being searched at once.") + + +@st.dialog("Getting an OpenRouter API key", width="large") +def show_api_key_guidance(): + st.markdown( + """ +Protocol generation calls a language model through **OpenRouter**, which +provides access to models from many vendors β€” Anthropic, OpenAI, Google, Meta, +DeepSeek, Mistral and others β€” through a single key. Nothing else in this +application requires a key: prediction and optimisation run entirely locally. + +**To obtain a key** + +1. Go to **openrouter.ai** and create an account. +2. Open **Keys** from the account menu and choose **Create Key**. +3. Copy the key and paste it into the sidebar field. It is held only for this + browser session and is never stored or logged. + +**Free models.** Models whose name ends in `:free` cost nothing and need no +credit. They are the right place to start, but they are shared and heavily +rate-limited: a request may be refused outright when the model is busy, and the +remedy is to wait a minute or pick another. For routine use, add a small amount +of credit under **Credits**; a protocol typically costs well under one cent on +the mid-range models. + +**Choosing a model.** The menu lists a curated selection with a note on each, +filtered against OpenRouter's live catalogue so that a model withdrawn since +this release does not appear. Stronger reasoning models follow the protocol's +constraints more reliably β€” particularly the instruction to report an +implausible parameter rather than quietly correct it, and the instruction not +to invent supplier names or catalogue numbers. + """) + + +# ── sidebar ────────────────────────────────────────────────────────────────── + +root = models_root() +if root is None: + st.error( + "No model artefacts found. Expected `deploy/models/` beside this file " + "or at the repository root. Run `python 06_webapp/export_deployment.py` " + "to create them.") + st.stop() + +st.sidebar.header("Optimisation weights") +w_cell_pct = st.sidebar.slider( + "Cell-response weight (%)", 0, 100, 70, 5, + help="Printability weight is the remainder.") +w_print_pct = 100 - w_cell_pct +st.sidebar.number_input("Printability weight (%)", value=w_print_pct, + disabled=True) +if st.sidebar.button("What is WSSQ?", use_container_width=True): + show_wssq_guidance() + +st.sidebar.header("Models") +print_models = available_models("printability") +cell_models = available_models("cell_response") +if not print_models or not cell_models: + st.error("Model artefacts are present but none could be loaded.") + st.stop() + +def default_index(entries: list[dict]) -> int: + """ + Which model the menu opens on. + + The highest-ranked model overall when a GPU is present. On a CPU host - a + free Hugging Face Space, or most laptops - the highest-ranked model that is + not a foundation model, because an in-context model re-reads its whole + context on every pass and turns a search that takes seconds into one taking + minutes. The foundation models stay in the menu with their cost stated; the + difference is only what a first-time visitor is given before choosing. + """ + if serving.device() == "cuda": + return 0 + return next((i for i, e in enumerate(entries) + if e["family"] != "foundation"), 0) + + +print_entry = st.sidebar.selectbox( + "Printability model", print_models, format_func=model_label, + index=default_index(print_models)) +cell_entry = st.sidebar.selectbox( + "Cell-response model", cell_models, format_func=model_label, + index=default_index(cell_models)) +if "foundation" in (print_entry["family"], cell_entry["family"]): + st.sidebar.caption( + f"Foundation model selected - {print_entry['cost']}. These carry the " + f"corpus rather than fitted parameters and re-read it on every pass, " + f"so a search is slower than with a conventional classifier and much " + f"slower without a GPU.") +if st.sidebar.button("Model performance", use_container_width=True): + show_model_guidance() + +st.sidebar.header("Search") +n_trials = st.sidebar.number_input("Optimisation trials", 20, 2000, 150, 10) +if st.sidebar.button("How many trials?", use_container_width=True): + show_trial_guidance() + +st.sidebar.header("Protocol generation") +# Never prefilled from the server's own key. `type="password"` masks a value +# on screen but still sends it to the browser, so prefilling would hand the +# operator's key to every visitor of a public deployment. A key configured on +# the server remains usable - `proto.api_key` falls back to it when this field +# is empty - but it is never transmitted. +api_key_input = st.sidebar.text_input( + "OpenRouter API key", type="password", + help="Needed only for protocol generation. Held for this browser session " + "only; never stored or logged.") +if not api_key_input and proto.api_key(): + st.sidebar.caption("A key is configured on the server; leave this blank " + "to use it.") +llm_models, llm_verified = list_llms() +model_labels = [f"{m} Β· {tier}" for m, _, tier, _ in llm_models] +llm_idx = st.sidebar.selectbox( + "Language model", range(len(model_labels)), + format_func=lambda i: model_labels[i], + index=next((i for i, m in enumerate(llm_models) + if m[0] == proto.DEFAULT_MODEL), 0)) +llm_model, _vendor, _tier, llm_note = llm_models[llm_idx] +st.sidebar.caption( + llm_note if llm_verified + else f"{llm_note} \n_Availability unconfirmed: the OpenRouter " + f"catalogue could not be reached._") +if st.sidebar.button("How to get a key", use_container_width=True): + show_api_key_guidance() + +# ── main ───────────────────────────────────────────────────────────────────── + +st.title("MLATE: Machine Learning Applications in Tissue Engineering") +st.markdown( + "Define the ranges you can work within, and the optimiser searches them for " + "the formulation with the highest predicted scaffold quality. Predictions " + "come from models trained on 2,646 scaffold records extracted from the " + "literature, and are decision support rather than validated outcomes.") + +if "bio_rows" not in st.session_state: + st.session_state.bio_rows = [ + {"mat": "Alginate (%w/v)", "min": 1.0, "max": 6.0, "step": 0.5}] + +st.subheader("Biomaterials") +st.caption("Give a range for each component. The optimiser searches within it.") +c1, c2 = st.columns([1, 5]) +if c1.button("Add biomaterial"): + remaining = [m for m in BIOMATERIAL_OPTIONS + if m not in {r["mat"] for r in st.session_state.bio_rows}] + if remaining: + st.session_state.bio_rows.append( + {"mat": remaining[0], "min": 0.0, "max": 5.0, "step": 0.5}) +if c2.button("Clear all") and st.session_state.bio_rows: + st.session_state.bio_rows = [] + +for i, row in enumerate(list(st.session_state.bio_rows)): + a, b, c, d, e = st.columns([4, 1.4, 1.4, 1.4, 0.8]) + row["mat"] = a.selectbox("Material", BIOMATERIAL_OPTIONS, + index=BIOMATERIAL_OPTIONS.index(row["mat"]) + if row["mat"] in BIOMATERIAL_OPTIONS else 0, + key=f"mat{i}", label_visibility="collapsed") + obs = BIOMATERIAL_RANGES.get(row["mat"]) + row["min"] = b.number_input("min", value=float(row["min"]), step=0.1, + key=f"lo{i}") + row["max"] = c.number_input("max", value=float(row["max"]), step=0.1, + key=f"hi{i}") + row["step"] = d.number_input("step", value=float(row["step"]), step=0.1, + min_value=0.01, key=f"st{i}") + if e.button("βœ•", key=f"rm{i}"): + st.session_state.bio_rows.pop(i) + st.rerun() + if obs: + note = (f"observed in corpus: {obs['min']:.3g} – {obs['max']:.3g} " + f"(median {obs['median']:.3g}, n = {obs['n']})") + if row["max"] > obs["max"] or (row["min"] > 0 + and row["min"] < obs["min"]): + a.caption(f":orange[{note} β€” your range extends beyond this]") + else: + a.caption(note) + +st.markdown("---") +st.subheader("Cell line and density") +labels = [f"{c} ({CELL_LINE_COUNTS.get(c, 0)} records)" + for c in CELL_LINE_OPTIONS] +ci = st.selectbox("Cell line", range(len(CELL_LINE_OPTIONS)), + format_func=lambda i: labels[i], index=0) +cell_line = CELL_LINE_OPTIONS[ci] + +density_var = None +if cell_line == ACELLULAR_TOKEN: + st.info( + "Acellular mode. Cell response does not apply, so scaffold quality is " + "scored on printability alone and the cell-response weight is ignored.") +else: + obs = CELL_DENSITY_RANGES.get(cell_line) + d1, d2, d3 = st.columns(3) + dmin = d1.number_input("Density min (Γ—10⁢ cells/mL)", + value=float(obs["min"]) if obs else 1.0, step=0.5) + dmax = d2.number_input("Density max (Γ—10⁢ cells/mL)", + value=float(obs["max"]) if obs else 10.0, step=0.5) + dstep = d3.number_input("Density step", value=0.5, step=0.1, + min_value=0.01) + density_var = opt.Variable(cfg.CELL_COLS[1], dmin, dmax, dstep) + if obs: + st.caption(f"observed for {cell_line}: {obs['min']:.3g} – " + f"{obs['max']:.3g} (median {obs['median']:.3g})") + +st.markdown("---") +st.subheader("Crosslinking") +xl_vars = [] +for p in CROSSLINK: + a, b, c = st.columns(3) + lo = a.number_input(f"{p} β€” min", value=0.0, step=5.0, key=f"xlo{p}") + hi = b.number_input(f"{p} β€” max", value=300.0, step=5.0, key=f"xhi{p}") + stp = c.number_input("step", value=5.0, step=1.0, min_value=0.01, + key=f"xst{p}") + xl_vars.append(opt.Variable(p, lo, hi, stp)) + +st.subheader("Printing parameters") +DEFAULTS = {"Extrusion Pressure (kPa)": (20.0, 200.0, 5.0), + "Nozzle Movement Speed (mm/s)": (1.0, 20.0, 0.5), + "Nozzle Diameter (Β΅m)": (100.0, 600.0, 10.0), + "Syringe Temperature (Β°C)": (18.0, 40.0, 0.5), + "Substrate Temperature (Β°C)": (4.0, 40.0, 0.5)} +pp_vars = [] +for p in PROTOCOL_PARAMS: + lo0, hi0, s0 = DEFAULTS[p] + a, b, c = st.columns(3) + lo = a.number_input(f"{p} β€” min", value=lo0, step=s0, key=f"plo{p}") + hi = b.number_input(f"{p} β€” max", value=hi0, step=s0, key=f"phi{p}") + stp = c.number_input("step", value=s0, step=0.1, min_value=0.01, + key=f"pst{p}") + pp_vars.append(opt.Variable(p, lo, hi, stp)) + +st.markdown("---") + +if st.button("Optimise scaffold quality", type="primary"): + if not st.session_state.bio_rows: + st.error("Add at least one biomaterial.") + st.stop() + + pre, feature_columns = load_preprocessor() + space = opt.SearchSpace( + cell_line=cell_line, + biomaterials=[opt.Variable(r["mat"], r["min"], r["max"], r["step"]) + for r in st.session_state.bio_rows], + printing=xl_vars + pp_vars, + cell_density=density_var) + + objective = opt.Objective( + space, pre, feature_columns, + load_model(print_entry["path"], print_entry["family"]), + load_model(cell_entry["path"], cell_entry["family"]), + print_weight=w_print_pct / 100, cell_weight=w_cell_pct / 100) + + bar = st.progress(0.0, text="Searching…") + + def report(done, total, best): + bar.progress(min(done / total, 1.0), + text=f"Trial {done}/{total} Β· best WSSQ {best:.1f}%") + + best, score, study = opt.optimise(objective, n_trials=int(n_trials), + progress=report) + bar.empty() + + detail = objective.evaluate(best) + corpus, groups = load_corpus() + st.session_state.update( + best_params=best, best_value=score, best_detail=detail, + opt_cell_line=cell_line, + best_distance=(None if corpus is None + else opt.distance_report(best, corpus))) + +if "best_params" in st.session_state: + best = st.session_state.best_params + detail = st.session_state.best_detail + + st.success(f"Best WSSQ: **{st.session_state.best_value:.1f}%**") + m1, m2, m3 = st.columns(3) + m1.metric("Expected printability", + f"{detail['expected_printability']:.2f}", help="Scale 0–3") + if st.session_state.opt_cell_line != ACELLULAR_TOKEN: + m2.metric("Expected cell response", + f"{detail['expected_cell_response']:.2f}", help="Scale 1–5") + m3.metric("Cell line", st.session_state.opt_cell_line) + + st.subheader("Optimised formulation") + tidy = pd.DataFrame( + {"Parameter": list(best), "Value": [f"{v:.4g}" for v in best.values()]}) + st.dataframe(tidy, use_container_width=True, hide_index=True) + + st.caption( + "Predicted values, not measurements. The expected scores are " + "probability-weighted averages over the predicted class distribution, " + "so a value between two classes reflects genuine model uncertainty.") + + dist = st.session_state.get("best_distance") + if dist is None: + st.info( + "The corpus reference table is not present, so how far this " + "formulation sits from published work could not be checked. Run " + "`python 06_webapp/build_app_data.py` to generate it.") + else: + d1, d2 = st.columns(2) + d1.metric("Parameters outside the observed range", + dist["n_out_of_range"], + help="Values no published study in the corpus reports. The " + "models are extrapolating for these.") + d2.metric("Distance to nearest published formulation", + f"{dist['nearest_neighbour_distance']:.3f}", + help="Scaled Euclidean distance. A formulation can sit " + "inside the observed range of every single variable and " + "still be a combination no one has attempted; this " + "number is what catches that.") + if dist["n_out_of_range"]: + with st.expander("Which parameters, and by how much"): + st.dataframe(pd.DataFrame([ + {"Parameter": k, "Proposed": f"{v['value']:.4g}", + "Observed range": f"{v['observed_min']:.4g} - " + f"{v['observed_max']:.4g}"} + for k, v in dist["out_of_range"].items()]), + use_container_width=True, hide_index=True) + + st.markdown("---") + st.subheader("Fabrication protocol") + user_inquiry = st.text_area( + "Constraints or equipment you must work with", + placeholder="e.g. only a 25 G nozzle is available; UV source is " + "fixed at 10 mW/cmΒ²", + key="user_inquiry") + + if st.button("Generate protocol"): + key = proto.api_key(api_key_input) + if not key: + st.error("Enter an OpenRouter API key in the sidebar first.") + st.stop() + + bio = {k: v for k, v in best.items() if k in BIOMATERIAL_OPTIONS} + printing = {k: v for k, v in best.items() if k in PRINT_PARAMS} + # The neighbours and the extrapolation report are what ground the + # prompt. Passing neither leaves the template asserting that no similar + # formulation exists and that the candidate is in range, and neither + # would have been checked. + corpus, groups = load_corpus() + neighbours = (None if corpus is None else + proto.nearest_formulations(best, corpus, groups, n=3)) + form = proto.Formulation( + cell_line=st.session_state.opt_cell_line, + biomaterials=bio, printing=printing, + cell_density=best.get(cfg.CELL_COLS[1]), + expected_printability=detail["expected_printability"], + expected_cell_response=detail["expected_cell_response"], + printability_proba=detail.get("printability_proba"), + cell_response_proba=detail.get("cell_response_proba"), + wssq=st.session_state.best_value, + neighbours=neighbours, + extrapolation=st.session_state.get("best_distance")) + + # Streamed rather than awaited. A free model takes between thirty + # seconds and two and a half minutes to write a protocol, and a + # spinner held for that long is indistinguishable from a hang. + st.markdown("## Fabrication procedure") + status = st.empty() + stream_area = st.empty() + status.info(f"Generating with {llm_model}…") + pieces: list[str] = [] + + def on_chunk(piece: str) -> None: + pieces.append(piece) + stream_area.markdown("".join(pieces)) + + def on_retry(attempt: int, reason: str | None) -> None: + status.warning( + f"Attempt {attempt} of {proto.DEFAULT_RETRIES}: {reason}") + + try: + out = proto.generate( + form, key=key, model=llm_model, + user_constraints=user_inquiry, + n_records=(len(corpus) if corpus is not None else 2646), + on_chunk=on_chunk, on_retry=on_retry) + except Exception as exc: + status.empty() + stream_area.empty() + st.error(f"Generation failed: {exc}") + st.stop() + + status.empty() + stream_area.markdown(out["protocol"]) + if out.get("truncated"): + st.warning( + "The model reached its output limit and the protocol is " + "incomplete. Generate again, or choose another model.") + with st.expander("Generation record"): + st.json({k: v for k, v in out.items() if k != "protocol"}) + st.download_button("Download protocol (Markdown)", + out["protocol"], + file_name="mlate_protocol.md") + +st.markdown("---") +st.caption( + "MLATE V3 β€” decision support for 3D-printed and bioprinted scaffolds. " + "Predictions narrow the experimental search space; they do not replace " + "experimental validation.") diff --git a/biomaterials.py b/biomaterials.py index 43240fa6b4f966b9fb354cfaf5270f3eb760828c..a8719f94e4313b97e60ace500149adfe78b5decc 100644 --- a/biomaterials.py +++ b/biomaterials.py @@ -1,124 +1,926 @@ +""" +Biomaterial vocabulary and observed concentration ranges. + +GENERATED by 06_webapp/build_app_data.py - do not edit by hand. +Regenerate after any change to the dataset or the benchmark tables. +""" + BIOMATERIAL_OPTIONS = [ - "Alginate (%w/v)", - "PVA-HA (%w/v)", - "CaSO4 (%w/v)", - "Na2HPO4 (%w/v)", - "Gelatin (%w/v)", - "GelMA (%w/v)", - "laponite (%w/v)", - "graphene oxide (%w/v)", - "hydroxyapatite (%w/v)", - "Hyaluronic_Acid (%w/v)", - "hyaluronan metacrylate (%w/v)", - "NorHA (%w/v)", - "Fibroin/Fibrinogen (%w/v)", - "Pluronic P-123 (%w/v)", - "Collagen (%w/v)", - "Chitosan (%w/v)", - "CS-AEMA (%w/v)", - "RGD (mM)", - "TCP (%w/v)", - "Gellan (%w/v)", - "bioactive glass (%w/v)", - "Nano/Methycellulose (%w/v)", - "PEGTA (%w/v)", - "PEGMA (%w/v)", - "PEGDA (%w/v)", - "Agarose (%w/v)", - " hyaluronic acid+ Ph moieties (%w/v)", - "matrigel (%w/v)", - "CaCl2(mM)", - "NaCl(mM)", - "BaCl2(mM)", - "SrCl2(mM)", - "CaCO3 (mM)", - "Genipin (%w/v)", - "PVA (%wt)", - "trans-glutaminase (%w/v)", - "alginate lyase (U/ml)", - "D-glucose (%w/v)", - "PLGA (%w/v)", - "vascular tissued-derived dECM (%w/v)", - "PEG-8-SH (mM)", - "Alginate dialdehyde (%w/v)", - "Alginate sulfate (%w/v)", - "RGD-modified alginate (%w/v)", - "poly(N-isopropylacrylamide) grafted hyaluronan (%w/v)", - "chondroitin sulfate methacrylate (%w/v)", - "PCL (%w/v)", - "alginate methacrylate (%w/v)", - "HRP (U/ml)", - "Pluronic F127 (%w/v)/Lutrol F127 (%w/v)", - "Irgacure 2959 (%w/v)", - "Eosin Y (%w/v)", - "Ruthenium (mM)", - "sodium persulfate (SPS) (mM)", - "HEPES (mM)", - "LAP (%w/v)", - "glutaraldehyde (%w/v)", - "PBS (M)", - "glycerol (%w/v)", - "cECM (%w/v)", - "gel-fu(%w/v)", - "Rose Bengal (%w/v)", - "Vitamin B2(%w/v)", - "VEGF(%w/v)", - "Polypyrrole:PSS(%w/v)", - "boratebioactiveglass(%w/v)", - "astaxanthin(%w/v)", - "PRP (%v/v)", - "methacrylated collagen (%w/v)", - "Ξ±-Toc (Β΅M)", - "ascorbic acid (mM)", - "Liver dECM(%w/v)", - "galactosylated alginate (%w/v)", - "SC-PEG(%w/v)", - "SFMA-L(%w/v)", - "SFMA-M(%w/v)", - "SFMA-H(%w/v)", - "KdECMMA(%w/v)", - "BA silk fibronin (%w/v)", - "Carrageenan(%v)", - "Carbopol ETD 2020 NF (%w/v)", - "Carbopol Ultrez 10 NF(%w/v)", - "Carbopol NF-980(%w/v)", - "FBS (%v/v)", - "MeTro (%w/v)", - "Triethanolamine (%v/v)", - "PEG-Fibrinogen (%w/v)", - "polyethylene glycol dimethacrylate (%w/v)", - "aprotinin (Β΅g/ml)", - "gold nanorod (mg/mL)", - "egg white (w/v)", - "1-Vinyl-2-Pyrrolidione (v/v)", - "carboxyl functionalized carbon nanotubes (%w/v)", - "polyHIPE (%w/v)", - "Ξ²-D galactose (mM)", - "hydrogen peroxide (H2O2) (%v/v)", - "lactic acid v/v", - "NorCol (%w/v)", - "DDT (%w/v)", - "ammonium persulfate (mM)", - "diTyr-RGD (mM)", - "PHEG-Tyr (%w/v)", - "MMP2-degradable peptide (%w/v)", - "KdECM (%w/v)", - "EDC (mg)", - "NHS (mg)", - "VA086 (%w/v)", - "PGS (%w/v)", - "thiolated HA (%w/v)", - "boron nitride nanotubes (%w/v)", - "PEDOT:PSS (ul)", - "KCl (mM)", - "skeletal muscle ECM methacrylate (%w/v)", - "PEO (%w/v)", - "Carbon dots (mg/ml)", - "Laminin (ug/ml)", - "DF-PEG (%w/v)", - "omenta ECM (%w/v)", - "thrombin (unit/ml)", - "Carbon nanotube (CNT) (w/v)", - "Phytagel(%v)", - "Laponite-XLG (%w/w)" -] \ No newline at end of file + 'Alginate (%w/v)', + 'PVA-HA (%w/v)', + 'CaSO4 (%w/v)', + 'Na2HPO4 (%w/v)', + 'Gelatin (%w/v)', + 'GelMA (%w/v)', + 'laponite (%w/v)', + 'graphene oxide (%w/v)', + 'hydroxyapatite (%w/v)', + 'Hyaluronic_Acid (%w/v)', + 'hyaluronan methacrylate (%w/v)', + 'NorHA (%w/v)', + 'Fibroin/Fibrinogen (%w/v)', + 'Pluronic P-123 (%w/v)', + 'Collagen (%w/v)', + 'Chitosan (%w/v)', + 'CS-AEMA (%w/v)', + 'RGD (mM)', + 'TCP (%w/v)', + 'Gellan (%w/v)', + 'bioactive glass (%w/v)', + 'Nano/Methylcellulose (%w/v)', + 'PEGTA (%w/v)', + 'PEGMA (%w/v)', + 'PEGDA (%w/v)', + 'Agarose (%w/v)', + 'hyaluronic acid+ Ph moieties (%w/v)', + 'matrigel (%w/v)', + 'CaCl2 (mM)', + 'NaCl (mM)', + 'BaCl2 (mM)', + 'SrCl2 (mM)', + 'CaCO3 (mM)', + 'Genipin (%w/v)', + 'PVA (%wt)', + 'trans-glutaminase (%w/v)', + 'alginate lyase (U/ml)', + 'D-glucose (%w/v)', + 'PLGA (%w/v)', + 'vascular tissue-derived dECM (%w/v)', + 'PEG-8-SH (mM)', + 'Alginate dialdehyde (%w/v)', + 'Alginate sulfate (%w/v)', + 'RGD-modified alginate (%w/v)', + 'poly(N-isopropylacrylamide) grafted hyaluronan (%w/v)', + 'chondroitin sulfate methacrylate (%w/v)', + 'PCL (%w/v)', + 'alginate methacrylate (%w/v)', + 'HRP (U/ml)', + 'Pluronic F127 / Lutrol F127 (%w/v)', + 'Irgacure 2959 (%w/v)', + 'Eosin Y (%w/v)', + 'Ruthenium (mM)', + 'sodium persulfate (SPS) (mM)', + 'HEPES (mM)', + 'LAP (%w/v)', + 'glutaraldehyde (%w/v)', + 'PBS (M)', + 'glycerol (%w/v)', + 'cECM (%w/v)', + 'gel-fu (%w/v)', + 'Rose Bengal (%w/v)', + 'Vitamin B2 (%w/v)', + 'VEGF (%w/v)', + 'Polypyrrole:PSS (%w/v)', + 'borate bioactive glass (%w/v)', + 'astaxanthin (%w/v)', + 'PRP (%v/v)', + 'methacrylated collagen (%w/v)', + 'Ξ±-Toc (Β΅M)', + 'ascorbic acid (mM)', + 'Liver dECM (%w/v)', + 'galactosylated alginate (%w/v)', + 'SC-PEG (%w/v)', + 'SFMA-L (%w/v)', + 'SFMA-M (%w/v)', + 'SFMA-H (%w/v)', + 'KdECMMA (%w/v)', + 'BA silk fibroin (%w/v)', + 'Carrageenan (%v)', + 'Carbopol ETD 2020 NF (%w/v)', + 'Carbopol Ultrez 10 NF (%w/v)', + 'Carbopol NF-980 (%w/v)', + 'FBS (%v/v)', + 'MeTro (%w/v)', + 'Triethanolamine (%v/v)', + 'PEG-Fibrinogen (%w/v)', + 'polyethylene glycol dimethacrylate (%w/v)', + 'aprotinin (Β΅g/ml)', + 'gold nanorod (mg/mL)', + 'egg white (w/v)', + '1-Vinyl-2-pyrrolidone (v/v)', + 'carboxyl functionalized carbon nanotubes (%w/v)', + 'polyHIPE (%w/v)', + 'Ξ²-D galactose (mM)', + 'hydrogen peroxide (H2O2) (%v/v)', + 'lactic acid (v/v)', + 'NorCol (%w/v)', + 'DTT (%w/v)', + 'ammonium persulfate (mM)', + 'diTyr-RGD (mM)', + 'PHEG-Tyr (%w/v)', + 'MMP2-degradable peptide (%w/v)', + 'KdECM (%w/v)', + 'EDC (mg)', + 'NHS (mg)', + 'VA086 (%w/v)', + 'PGS (%w/v)', + 'thiolated HA (%w/v)', + 'boron nitride nanotubes (%w/v)', + 'PEDOT:PSS (ul)', + 'KCl (mM)', + 'skeletal muscle ECM methacrylate (%w/v)', + 'PEO (%w/v)', + 'Carbon dots (mg/ml)', + 'Laminin (ug/ml)', + 'DF-PEG (%w/v)', + 'omentum ECM (%w/v)', + 'thrombin (unit/ml)', + 'Carbon nanotube (CNT) (w/v)', + 'Phytagel (%v)', + 'Laponite-XLG (%w/w)', + 'sodium carboxymethyl cellulose (mg)', + 'lysozyme amyloid nanofibrils:gold nanoparticles (mg)', + 'Lysozyme amyloid nanofibrils (mg)', + 'rGO (mg/ml)', + 'Methacrylated gellan gum (%w/v)', + 'Acetylsalicylic Acid (%w/w)', + 'PVA methacrylate (%w/v)', + 'MXene (mg/ml)', +] + +# Observed range among formulations that CONTAIN each material. +# The UI seeds its range widgets from these so a user starts +# inside the region the models were trained on; values outside +# are permitted but flagged as extrapolation. +BIOMATERIAL_RANGES = { + "Alginate (%w/v)": { + "min": 0.25, + "max": 20.0, + "median": 4.0, + "n": 1138 + }, + "PVA-HA (%w/v)": { + "min": 0.3, + "max": 2.5, + "median": 2.0, + "n": 15 + }, + "CaSO4 (%w/v)": { + "min": 0.03, + "max": 25.71, + "median": 1.0, + "n": 32 + }, + "Na2HPO4 (%w/v)": { + "min": 0.12, + "max": 0.15, + "median": 0.15, + "n": 9 + }, + "Gelatin (%w/v)": { + "min": 0.5, + "max": 80.0, + "median": 5.0, + "n": 797 + }, + "GelMA (%w/v)": { + "min": 1.0, + "max": 30.0, + "median": 7.0, + "n": 880 + }, + "laponite (%w/v)": { + "min": 0.05, + "max": 2.3, + "median": 2.3, + "n": 96 + }, + "graphene oxide (%w/v)": { + "min": 0.001, + "max": 1.5, + "median": 0.5, + "n": 43 + }, + "hydroxyapatite (%w/v)": { + "min": 2.0, + "max": 70.0, + "median": 70.0, + "n": 43 + }, + "Hyaluronic_Acid (%w/v)": { + "min": 0.2, + "max": 2.0, + "median": 0.3, + "n": 62 + }, + "hyaluronan methacrylate (%w/v)": { + "min": 0.1, + "max": 6.0, + "median": 2.0, + "n": 151 + }, + "NorHA (%w/v)": { + "min": 2.0, + "max": 2.0, + "median": 2.0, + "n": 4 + }, + "Fibroin/Fibrinogen (%w/v)": { + "min": 0.009375, + "max": 25.0, + "median": 2.0, + "n": 207 + }, + "Pluronic P-123 (%w/v)": { + "min": 40.0, + "max": 60.0, + "median": 50.0, + "n": 12 + }, + "Collagen (%w/v)": { + "min": 0.01, + "max": 7.8, + "median": 2.0, + "n": 85 + }, + "Chitosan (%w/v)": { + "min": 1.0, + "max": 62.0, + "median": 2.0, + "n": 74 + }, + "CS-AEMA (%w/v)": { + "min": 4.0, + "max": 4.0, + "median": 4.0, + "n": 8 + }, + "RGD (mM)": { + "min": 3.0, + "max": 3.0, + "median": 3.0, + "n": 1 + }, + "TCP (%w/v)": { + "min": 0.5, + "max": 3.0, + "median": 0.5, + "n": 9 + }, + "Gellan (%w/v)": { + "min": 0.5, + "max": 150.0, + "median": 150.0, + "n": 54 + }, + "bioactive glass (%w/v)": { + "min": 1.0, + "max": 50.0, + "median": 1.0, + "n": 7 + }, + "Nano/Methylcellulose (%w/v)": { + "min": 0.25, + "max": 80.0, + "median": 4.0, + "n": 157 + }, + "PEGTA (%w/v)": { + "min": 1.0, + "max": 3.0, + "median": 2.0, + "n": 33 + }, + "PEGMA (%w/v)": { + "min": 12.5, + "max": 100.0, + "median": 17.5, + "n": 9 + }, + "PEGDA (%w/v)": { + "min": 1.0, + "max": 20.0, + "median": 8.25, + "n": 119 + }, + "Agarose (%w/v)": { + "min": 0.5, + "max": 60.0, + "median": 24.0, + "n": 51 + }, + "hyaluronic acid+ Ph moieties (%w/v)": { + "min": 0.1, + "max": 1.5, + "median": 0.8, + "n": 8 + }, + "matrigel (%w/v)": { + "min": 5.0, + "max": 50.0, + "median": 10.0, + "n": 19 + }, + "CaCl2 (mM)": { + "min": 0.001, + "max": 1500.0, + "median": 100.0, + "n": 1057 + }, + "NaCl (mM)": { + "min": 0.72, + "max": 350.0, + "median": 145.0, + "n": 23 + }, + "BaCl2 (mM)": { + "min": 55.0, + "max": 60.0, + "median": 55.0, + "n": 12 + }, + "SrCl2 (mM)": { + "min": 20.0, + "max": 70.0, + "median": 20.0, + "n": 6 + }, + "CaCO3 (mM)": { + "min": 15.88, + "max": 25.71, + "median": 18.0, + "n": 9 + }, + "Genipin (%w/v)": { + "min": 0.025, + "max": 1.0, + "median": 0.025, + "n": 18 + }, + "PVA (%wt)": { + "min": 3.0, + "max": 15.0, + "median": 15.0, + "n": 50 + }, + "trans-glutaminase (%w/v)": { + "min": 0.04, + "max": 6.0, + "median": 1.0, + "n": 63 + }, + "alginate lyase (U/ml)": { + "min": 0.5, + "max": 500.0, + "median": 5.0, + "n": 7 + }, + "D-glucose (%w/v)": { + "min": 4.4, + "max": 4.4, + "median": 4.4, + "n": 5 + }, + "PLGA (%w/v)": { + "min": 100.0, + "max": 100.0, + "median": 100.0, + "n": 4 + }, + "vascular tissue-derived dECM (%w/v)": { + "min": 1.0, + "max": 3.0, + "median": 3.0, + "n": 7 + }, + "PEG-8-SH (mM)": { + "min": 2.25, + "max": 8.0, + "median": 2.25, + "n": 10 + }, + "Alginate dialdehyde (%w/v)": { + "min": 2.0, + "max": 7.5, + "median": 3.75, + "n": 48 + }, + "Alginate sulfate (%w/v)": { + "min": 1.0, + "max": 1.0, + "median": 1.0, + "n": 26 + }, + "RGD-modified alginate (%w/v)": { + "min": 1.0, + "max": 1.0, + "median": 1.0, + "n": 2 + }, + "poly(N-isopropylacrylamide) grafted hyaluronan (%w/v)": { + "min": 15.0, + "max": 15.0, + "median": 15.0, + "n": 3 + }, + "chondroitin sulfate methacrylate (%w/v)": { + "min": 5.0, + "max": 5.0, + "median": 5.0, + "n": 1 + }, + "PCL (%w/v)": { + "min": 1.0, + "max": 100.0, + "median": 8.0, + "n": 34 + }, + "alginate methacrylate (%w/v)": { + "min": 1.0, + "max": 3.0, + "median": 3.0, + "n": 24 + }, + "HRP (U/ml)": { + "min": 5.0, + "max": 100.0, + "median": 15.0, + "n": 68 + }, + "Pluronic F127 / Lutrol F127 (%w/v)": { + "min": 3.0, + "max": 100.0, + "median": 6.0, + "n": 83 + }, + "Irgacure 2959 (%w/v)": { + "min": 0.01, + "max": 2.0, + "median": 0.25, + "n": 530 + }, + "Eosin Y (%w/v)": { + "min": 0.5, + "max": 100.0, + "median": 0.5, + "n": 32 + }, + "Ruthenium (mM)": { + "min": 0.254, + "max": 1.0, + "median": 0.5, + "n": 28 + }, + "sodium persulfate (SPS) (mM)": { + "min": 2.52, + "max": 10.0, + "median": 5.0, + "n": 28 + }, + "HEPES (mM)": { + "min": 10.0, + "max": 25.0, + "median": 10.0, + "n": 22 + }, + "LAP (%w/v)": { + "min": 0.01, + "max": 4.46, + "median": 0.4, + "n": 310 + }, + "glutaraldehyde (%w/v)": { + "min": 0.125, + "max": 0.4, + "median": 0.25, + "n": 62 + }, + "PBS (M)": { + "min": 0.082, + "max": 0.328, + "median": 0.165, + "n": 3 + }, + "glycerol (%w/v)": { + "min": 10.0, + "max": 10.0, + "median": 10.0, + "n": 21 + }, + "cECM (%w/v)": { + "min": 0.1, + "max": 20.0, + "median": 4.6, + "n": 118 + }, + "gel-fu (%w/v)": { + "min": 10.0, + "max": 155.0, + "median": 100.0, + "n": 15 + }, + "Rose Bengal (%w/v)": { + "min": 1.0, + "max": 5.0, + "median": 5.0, + "n": 14 + }, + "Vitamin B2 (%w/v)": { + "min": 0.02, + "max": 0.2, + "median": 0.02, + "n": 20 + }, + "VEGF (%w/v)": { + "min": 0.01, + "max": 10.0, + "median": 1.0, + "n": 6 + }, + "Polypyrrole:PSS (%w/v)": { + "min": 0.1, + "max": 0.4, + "median": 0.2, + "n": 3 + }, + "borate bioactive glass (%w/v)": { + "min": 0.1, + "max": 0.1, + "median": 0.1, + "n": 1 + }, + "astaxanthin (%w/v)": { + "min": 0.01, + "max": 0.01, + "median": 0.01, + "n": 1 + }, + "PRP (%v/v)": { + "min": 20.0, + "max": 20.0, + "median": 20.0, + "n": 3 + }, + "methacrylated collagen (%w/v)": { + "min": 0.2, + "max": 50.0, + "median": 0.48, + "n": 25 + }, + "\u03b1-Toc (\u00b5M)": { + "min": 100.0, + "max": 100.0, + "median": 100.0, + "n": 5 + }, + "ascorbic acid (mM)": { + "min": 3.4, + "max": 3.4, + "median": 3.4, + "n": 5 + }, + "Liver dECM (%w/v)": { + "min": 0.5, + "max": 100.0, + "median": 2.0, + "n": 139 + }, + "galactosylated alginate (%w/v)": { + "min": 0.375, + "max": 1.0, + "median": 1.0, + "n": 22 + }, + "SC-PEG (%w/v)": { + "min": 1.44, + "max": 1.44, + "median": 1.44, + "n": 4 + }, + "SFMA-L (%w/v)": { + "min": 10.0, + "max": 10.0, + "median": 10.0, + "n": 1 + }, + "SFMA-M (%w/v)": { + "min": 10.0, + "max": 10.0, + "median": 10.0, + "n": 1 + }, + "SFMA-H (%w/v)": { + "min": 10.0, + "max": 10.0, + "median": 10.0, + "n": 1 + }, + "KdECMMA (%w/v)": { + "min": 1.0, + "max": 3.0, + "median": 2.0, + "n": 15 + }, + "BA silk fibroin (%w/v)": { + "min": 0.5, + "max": 3.0, + "median": 1.5, + "n": 32 + }, + "Carrageenan (%v)": { + "min": 0.5, + "max": 1.5, + "median": 1.0, + "n": 43 + }, + "Carbopol ETD 2020 NF (%w/v)": { + "min": 0.1, + "max": 1.2, + "median": 0.5, + "n": 32 + }, + "Carbopol Ultrez 10 NF (%w/v)": { + "min": 1.5, + "max": 1.5, + "median": 1.5, + "n": 6 + }, + "Carbopol NF-980 (%w/v)": { + "min": 1.2, + "max": 1.2, + "median": 1.2, + "n": 4 + }, + "FBS (%v/v)": { + "min": 10.0, + "max": 10.0, + "median": 10.0, + "n": 34 + }, + "MeTro (%w/v)": { + "min": 7.5, + "max": 7.5, + "median": 7.5, + "n": 28 + }, + "Triethanolamine (%v/v)": { + "min": 3.0, + "max": 3.0, + "median": 3.0, + "n": 4 + }, + "PEG-Fibrinogen (%w/v)": { + "min": 1.0, + "max": 1.0, + "median": 1.0, + "n": 8 + }, + "polyethylene glycol dimethacrylate (%w/v)": { + "min": 1.0, + "max": 1.0, + "median": 1.0, + "n": 50 + }, + "aprotinin (\u00b5g/ml)": { + "min": 0.2, + "max": 10.0, + "median": 0.2, + "n": 3 + }, + "gold nanorod (mg/mL)": { + "min": 0.1, + "max": 0.1, + "median": 0.1, + "n": 24 + }, + "egg white (w/v)": { + "min": 1.0, + "max": 3.0, + "median": 2.0, + "n": 10 + }, + "1-Vinyl-2-pyrrolidone (v/v)": { + "min": 0.75, + "max": 0.75, + "median": 0.75, + "n": 4 + }, + "carboxyl functionalized carbon nanotubes (%w/v)": { + "min": 0.3, + "max": 2.0, + "median": 1.0, + "n": 9 + }, + "polyHIPE (%w/v)": { + "min": 1.0, + "max": 5.0, + "median": 3.0, + "n": 5 + }, + "\u03b2-D galactose (mM)": { + "min": 40.0, + "max": 40.0, + "median": 40.0, + "n": 2 + }, + "hydrogen peroxide (H2O2) (%v/v)": { + "min": 0.09, + "max": 3.87, + "median": 3.87, + "n": 55 + }, + "lactic acid (v/v)": { + "min": 3.0, + "max": 3.0, + "median": 3.0, + "n": 30 + }, + "NorCol (%w/v)": { + "min": 0.2, + "max": 1.0, + "median": 0.6, + "n": 13 + }, + "DTT (%w/v)": { + "min": 0.5, + "max": 8.0, + "median": 0.7, + "n": 10 + }, + "ammonium persulfate (mM)": { + "min": 3.75, + "max": 5.0, + "median": 4.375, + "n": 16 + }, + "diTyr-RGD (mM)": { + "min": 2.0, + "max": 6.0, + "median": 4.0, + "n": 8 + }, + "PHEG-Tyr (%w/v)": { + "min": 10.0, + "max": 10.0, + "median": 10.0, + "n": 12 + }, + "MMP2-degradable peptide (%w/v)": { + "min": 0.1, + "max": 0.5, + "median": 0.3, + "n": 8 + }, + "KdECM (%w/v)": { + "min": 1.0, + "max": 5.0, + "median": 3.3, + "n": 11 + }, + "EDC (mg)": { + "min": 50.0, + "max": 50.0, + "median": 50.0, + "n": 22 + }, + "NHS (mg)": { + "min": 10.0, + "max": 25.0, + "median": 10.0, + "n": 22 + }, + "VA086 (%w/v)": { + "min": 0.5, + "max": 20.0, + "median": 0.75, + "n": 10 + }, + "PGS (%w/v)": { + "min": 2.0, + "max": 20.0, + "median": 11.0, + "n": 2 + }, + "thiolated HA (%w/v)": { + "min": 0.04, + "max": 0.067, + "median": 0.05, + "n": 10 + }, + "boron nitride nanotubes (%w/v)": { + "min": 0.05, + "max": 0.1, + "median": 0.075, + "n": 3 + }, + "PEDOT:PSS (ul)": { + "min": 0.1, + "max": 0.3, + "median": 0.1, + "n": 18 + }, + "KCl (mM)": { + "min": 5.0, + "max": 5.0, + "median": 5.0, + "n": 15 + }, + "skeletal muscle ECM methacrylate (%w/v)": { + "min": 3.0, + "max": 3.0, + "median": 3.0, + "n": 7 + }, + "PEO (%w/v)": { + "min": 0.4, + "max": 0.4, + "median": 0.4, + "n": 9 + }, + "Carbon dots (mg/ml)": { + "min": 20.0, + "max": 20.0, + "median": 20.0, + "n": 9 + }, + "Laminin (ug/ml)": { + "min": 10.0, + "max": 93.75, + "median": 70.0, + "n": 13 + }, + "DF-PEG (%w/v)": { + "min": 25.0, + "max": 25.0, + "median": 25.0, + "n": 6 + }, + "omentum ECM (%w/v)": { + "min": 2.0, + "max": 2.0, + "median": 2.0, + "n": 2 + }, + "thrombin (unit/ml)": { + "min": 2.5, + "max": 50.0, + "median": 10.0, + "n": 41 + }, + "Carbon nanotube (CNT) (w/v)": { + "min": 0.5, + "max": 6.0, + "median": 1.5, + "n": 18 + }, + "Phytagel (%v)": { + "min": 0.3, + "max": 0.3, + "median": 0.3, + "n": 1 + }, + "Laponite-XLG (%w/w)": { + "min": 2.3, + "max": 2.3, + "median": 2.3, + "n": 25 + }, + "sodium carboxymethyl cellulose (mg)": { + "min": 75.0, + "max": 150.0, + "median": 100.0, + "n": 60 + }, + "lysozyme amyloid nanofibrils:gold nanoparticles (mg)": { + "min": 2.5, + "max": 37.5, + "median": 12.5, + "n": 6 + }, + "Lysozyme amyloid nanofibrils (mg)": { + "min": 2.5, + "max": 37.5, + "median": 12.5, + "n": 31 + }, + "rGO (mg/ml)": { + "min": 150.0, + "max": 150.0, + "median": 150.0, + "n": 2 + }, + "Methacrylated gellan gum (%w/v)": { + "min": 2.0, + "max": 3.5, + "median": 3.0, + "n": 15 + }, + "Acetylsalicylic Acid (%w/w)": { + "min": 10.0, + "max": 10.0, + "median": 10.0, + "n": 3 + }, + "PVA methacrylate (%w/v)": { + "min": 1.0, + "max": 5.0, + "median": 5.0, + "n": 11 + }, + "MXene (mg/ml)": { + "min": 0.1, + "max": 5.0, + "median": 0.75, + "n": 6 + } +} diff --git a/cell_lines.py b/cell_lines.py index 08b60291e75680b3c16061f4fd2ea75d9fe7e657..fe4eba5e46221b1fc6aeb10e82b2bdd51700837c 100644 --- a/cell_lines.py +++ b/cell_lines.py @@ -1,178 +1,1320 @@ +""" +Cell-line vocabulary and observed seeding densities. + +GENERATED by 06_webapp/build_app_data.py - do not edit by hand. +Regenerate after any change to the dataset or the benchmark tables. +""" + +# Ordered by frequency in the corpus, with the acellular token +# first because it selects a different mode: cell response is +# not applicable and WSSQ falls back to printability alone. +ACELLULAR_TOKEN = 'NoCellCultured' + CELL_LINE_OPTIONS = [ - "NoCellCultured", - "chondrocyteyte", - "HepG2", - "bMSCs", - "HUVECs", - "NIH3T3", - "MESCs", - "hiPSC-CMs /ATCCs", - "CPCs", - "L929", - "Myoblast cells", - "hiPSCs", - "HepaRG", - "hESCs", - "10T1/2", - "Cardiac progenitor cells", - "NSCLC PDX", - "RAMECs", - "hASCs", - "HAVIC", - "Primary mouse hepatocyte", - "PTECs", - "human nasoseptal chondrocytes", - "PDX", - "HPFs", - "U87-MG", - "ESCs", - "HASSMC", - "dermal fibroblasts", - "MC3T3-E1", - "Schwann cells", - "hiPSC-CMs and HS-27A", - "Saos-2 ", - "SU3", - "hTMSCs", - "HACs", - "HADMSCs", - "HeLa", - "human primary kidney cells", - "myoblasts", - "MSCs", - "human primary kidneycells", - "293FT", - "HEK 293FT", - "Wnt3a-293FT", - "RSC96/HUVECs", - "human adipogenic mesenchymal stem cells", - "HEPG2/ECs", - "HUVECs/MSCs", - "RHECs", - "Human non-small cell lung cancer line Calu-3 (Calu-3)", - "HL-1", - "mouse cardiac cells", - "IMR-90", - "EPCs", - "MRC5", - "rMSC", - "basil plant cell", - "hNCs", - "A549", - "human induced pluripotent stem cell-derived cardiomyocytes", - "bMSCs/hACs", - "EA.hy 926 cells", - "HepG2/C3A", - "human epithelial lung carcinoma cells", - "Human cardiac fibroblasts", - "hTERT-MSC", - "cardiomyocytes", - "Huh7", - "NRCMs", - "HCF", - "Wnt reporter-293FT", - "neonatal rat ventricular CFs", - "human coronary artery endothelial cells", - "hiPSC-CM / fibroblasts", - "primary mouse hepatocyte", - "NIH3T3/ HUVECs", - "murine macrophage-like cell line", - "Endothelial cells", - "Human aortic VIC", - "sADSC", - "HUVECs/H9C2", - "neonatal rat ventricular cardiomyocytes", - "MG-63", - "Neonatal mouse cardiomyocytes (NMVCMs)", - "human hepatic stellate cell line", - "HEK-293", - "aHSC", - "MFCs", - "fibroblasts", - "HNDF", - "cardiomyocyte/MSCs", - "ADSCs", - "HCASMCs", - "cardiomyocyte", - "hCPCs", - "Human CM /adult human fibroblasts ", - "primary rat hepatocyte", - "human cardiac progenitor cells", - "SMC", - "Human MSCs", - "ACPCs", - "Huh7/HepaRG", - "Human umbilical vein endothelial cells", - "ATDC5", - "hESC-derived HLCs", - "NIH 3T3", - "n neonatal mouse ventricular cardiomyocytes", - "CFs/CMs/HUVECs", - "MRC-5", - "VIC", - "eHep", - "hUVECs/NIH3T3", - "MC3T3", - "HLC", - "hepatoma", - "FB", - "A549 GFP+", - "HPAAF", - "PMHs", - "HUVSMCs", - "Human CPCs", - "Fibroblasts/THP-1", - "rat ventricular cardiomyocytes", - "iCMs/iCFs/iECs", - "HUVEC/HHSC", - "Human CPCs / MSCs", - "HepaRG/LX-2 ", - "iCMs/iCFs/iECs/iCMFs", - "LX-2 ", - "SMMC-7721", - "Hepatoblast- single cell/iESC/iMSC", - "iCMFs", - "Hepatoblast- spheroid/iESC/iMSC", - "hBM-MSCs", - "BMSCs", - "HUVECs and HHSCs", - "Intrahepatic cholangiocarcinoma (ICC)", - "VICs", - "CMs/CFs", - "human neonatal dermal fi broblasts", - "10T1/2 fibroblast-laden cells", - "human cardiac fibroblasts", - "neonatal rat ventricular CMs", - "HUVECs and hiPSC-CS", - "iPSC-derived CM", - "Human Umbilical Vein Endothelial Cells + iPSC-derived CM", - "rabbit bone marrow mesenchymal stem cells", - "Neonatal rat cardiomyocytes", - "NIH 3T3 mouse fibroblasts", - "Human CPCs & MSCs", - "NSCLC PDX/CAFs", - "hiPSCs-derived HLCs", - "U87", - "75% hepatoblast cells, 20% iEC and 5% iMSC", - "SMCs", - "A549/95-D cells", - "NCI-H441", - "pancreatic cancer cell", - "prostate cancer stem cell", - "human primary parathyroid cells ", - "primary human hepatocytes", - "CPCs / MSCs", - "CPCs ", - "cardiac fibroblasts", - "iPSCs-derived cardiomyocytes", - "cardiomyocytes/ fibroblasts", - "hiPSC-CM", - "iPSCs/HUVECs", - "iPSC-CMs", - "iPSCs", - "H1395", - "PC9", - "H1650", - "HULEC-5a", - "NCI-H1703" -] \ No newline at end of file + 'NoCellCultured', + 'HUVECs', + 'HepG2', + 'bMSCs', + 'chondrocytes', + 'NIH3T3', + 'HepaRG', + 'hiPSC-CMs /ATCCs', + 'Myoblast cells', + 'H9c2', + 'HEK', + 'CPCs', + 'MESCs', + 'hiPSCs', + 'L929', + 'Primary mouse hepatocyte', + 'NSCLC PDX', + 'human primary kidney cells', + 'Cardiac progenitor cells', + '10T1/2', + 'HAVIC', + 'RAMECs', + 'hESCs', + 'human nasoseptal chondrocytes', + 'PDX', + 'PTECs', + 'HPFs', + 'MG-63', + 'dermal fibroblasts', + 'HASSMC', + 'Saos-2', + 'MC3T3-E1', + 'hASCs', + 'SU3', + 'ESCs', + 'RPTEC/TERT1', + 'hiPSC-CMs and HS-27A', + 'HADMSCs', + 'HACs', + 'MRC5', + 'HeLa', + 'RHECs', + 'cardiomyocytes', + 'human adipogenic mesenchymal stem cells', + 'MSCs', + 'HUVECs/MSCs', + 'U87-MG', + 'RSC96/HUVECs', + 'HEPG2/ECs', + 'A549', + 'Human non-small cell lung cancer line Calu-3 (Calu-3)', + 'hTMSCs', + 'mouse cardiac cells', + 'Human cardiac fibroblasts', + 'Schwann cells', + 'hTERT-MSC', + 'bMSCs/hACs', + 'basil plant cell', + 'EPCs', + 'HepaRG / LX-2', + 'Caki-1', + 'human induced pluripotent stem cell-derived cardiomyocytes', + 'human epithelial lung carcinoma cells', + 'HCF', + 'Huh7', + '3T3 Fibroblasts', + 'EA.hy 926 cells', + 'human coronary artery endothelial cells', + 'NIH3T3/ HUVECs', + 'neonatal rat ventricular CFs', + 'HepG2/C3A', + 'IMR-90', + 'rMSC', + 'hiPSC-CM / fibroblasts', + 'HUVECs/H9C2', + 'fibroblasts', + 'Endothelial cells', + 'neonatal rat ventricular cardiomyocytes', + 'Neonatal mouse cardiomyocytes (NMVCMs)', + 'murine macrophage-like cell line', + '293FT', + 'sADSC', + 'human hepatic stellate cell line', + 'aHSC', + 'NRCMs', + 'HBE', + 'HEK-293', + 'Wnt3a-293FT', + 'Human aortic VIC', + 'HEK 293FT', + 'HNDF', + 'Huh7/HepaRG', + 'hiHeps', + 'hCPCs', + 'pancreatic islets cells', + 'ICC', + 'Wnt reporter-293FT', + 'Human CPCs', + 'human cardiac progenitor cells', + 'SMC', + 'hepatoma', + 'cardiomyocyte/MSCs', + 'FB', + 'eHep', + 'PMHs', + 'Human CM /adult human fibroblasts', + 'HUVSMCs', + 'ACPCs', + 'MC3T3', + 'hNCs', + 'primary rat hepatocyte', + 'Human umbilical vein endothelial cells', + 'HPAAF', + 'VIC', + 'HLC', + 'ADSCs', + 'MFCs', + 'A549 GFP+', + 'ATDC5', + 'CFs/CMs/HUVECs', + 'n neonatal mouse ventricular cardiomyocytes', + 'Human MSCs', + 'Human CPCs / MSCs', + 'hESC-derived HLCs', + 'HCASMCs', + 'iCMs/iCFs/iECs/iCMFs', + 'VICs', + 'iPSCs', + 'primary human hepatocytes', + 'SMCs', + 'iCMs/iCFs/iECs', + 'PC9', + 'H1395', + 'H1650', + 'HUVECs and hiPSC-CS', + 'human neonatal dermal fi broblasts', + '10T1/2 fibroblast-laden cells', + 'iCMFs', + 'neonatal rat ventricular CMs', + 'U87', + 'NSCLC PDX/CAFs', + 'iPSC-derived CM', + 'NIH 3T3 mouse fibroblasts', + 'Neonatal rat cardiomyocytes', + 'hiPSCs-derived HLCs', + 'iPSCs/HUVECs', + 'HULEC-5a', + 'HUVECs and HHSCs', + 'NCI-H1703', + '75% hepatoblast cells, 20% iEC and 5% iMSC', + 'Intrahepatic cholangiocarcinoma (ICC)', + 'NCI-H441', + 'Human Umbilical Vein Endothelial Cells + iPSC-derived CM', + 'rabbit bone marrow mesenchymal stem cells', + 'CMs/CFs', + 'cardiac fibroblasts', + 'iPSCs-derived cardiomyocytes', + 'cardiomyocytes/ fibroblasts', + 'hiPSC-CM', + 'iPSC-CMs', + 'hBM-MSCs', + 'Hepatoblast- single cell/iESC/iMSC', + 'Hepatoblast- spheroid/iESC/iMSC', + 'human primary parathyroid cells', + 'prostate cancer stem cell', + 'HL-1', + 'CPCs / MSCs', + 'pancreatic cancer cell', + 'A549/95-D cells', + 'SMMC-7721', + 'rat ventricular cardiomyocytes', + 'A549 lung adenocarcinoma cells', + 'HMEC-1/ fibroblast/ THP-1', + 'HMEC-1', + 'HAVSMC', + 'HUVEC/HHSC', + 'LX-2', + 'Fibroblasts/THP-1', + 'HLF', + 'HUVECs / LX-2', + 'CCAO', + 'Hepa RG/ HUVEC', + 'BEAS2B', + 'PDGFRΞ²+', + 'hMSCs', + 'CiPSC-HOs', + 'tubular epithelial', +] + +CELL_LINE_COUNTS = { + 'NoCellCultured': 1573, + 'HUVECs': 75, + 'HepG2': 71, + 'bMSCs': 65, + 'chondrocytes': 39, + 'NIH3T3': 35, + 'HepaRG': 28, + 'hiPSC-CMs /ATCCs': 26, + 'Myoblast cells': 26, + 'H9c2': 25, + 'HEK': 24, + 'CPCs': 21, + 'MESCs': 20, + 'hiPSCs': 17, + 'L929': 15, + 'Primary mouse hepatocyte': 14, + 'NSCLC PDX': 13, + 'human primary kidney cells': 13, + 'Cardiac progenitor cells': 13, + '10T1/2': 13, + 'HAVIC': 12, + 'RAMECs': 12, + 'hESCs': 11, + 'human nasoseptal chondrocytes': 10, + 'PDX': 10, + 'PTECs': 10, + 'HPFs': 10, + 'MG-63': 9, + 'dermal fibroblasts': 9, + 'HASSMC': 9, + 'Saos-2': 8, + 'MC3T3-E1': 8, + 'hASCs': 8, + 'SU3': 8, + 'ESCs': 8, + 'RPTEC/TERT1': 8, + 'hiPSC-CMs and HS-27A': 8, + 'HADMSCs': 7, + 'HACs': 7, + 'MRC5': 7, + 'HeLa': 7, + 'RHECs': 6, + 'cardiomyocytes': 6, + 'human adipogenic mesenchymal stem cells': 6, + 'MSCs': 6, + 'HUVECs/MSCs': 6, + 'U87-MG': 6, + 'RSC96/HUVECs': 6, + 'HEPG2/ECs': 6, + 'A549': 6, + 'Human non-small cell lung cancer line Calu-3 (Calu-3)': 5, + 'hTMSCs': 5, + 'mouse cardiac cells': 5, + 'Human cardiac fibroblasts': 5, + 'Schwann cells': 5, + 'hTERT-MSC': 4, + 'bMSCs/hACs': 4, + 'basil plant cell': 4, + 'EPCs': 4, + 'HepaRG / LX-2': 4, + 'Caki-1': 4, + 'human induced pluripotent stem cell-derived cardiomyocytes': 4, + 'human epithelial lung carcinoma cells': 4, + 'HCF': 4, + 'Huh7': 4, + '3T3 Fibroblasts': 4, + 'EA.hy 926 cells': 4, + 'human coronary artery endothelial cells': 4, + 'NIH3T3/ HUVECs': 4, + 'neonatal rat ventricular CFs': 4, + 'HepG2/C3A': 4, + 'IMR-90': 4, + 'rMSC': 4, + 'hiPSC-CM / fibroblasts': 3, + 'HUVECs/H9C2': 3, + 'fibroblasts': 3, + 'Endothelial cells': 3, + 'neonatal rat ventricular cardiomyocytes': 3, + 'Neonatal mouse cardiomyocytes (NMVCMs)': 3, + 'murine macrophage-like cell line': 3, + '293FT': 3, + 'sADSC': 3, + 'human hepatic stellate cell line': 3, + 'aHSC': 3, + 'NRCMs': 3, + 'HBE': 3, + 'HEK-293': 3, + 'Wnt3a-293FT': 3, + 'Human aortic VIC': 3, + 'HEK 293FT': 3, + 'HNDF': 2, + 'Huh7/HepaRG': 2, + 'hiHeps': 2, + 'hCPCs': 2, + 'pancreatic islets cells': 2, + 'ICC': 2, + 'Wnt reporter-293FT': 2, + 'Human CPCs': 2, + 'human cardiac progenitor cells': 2, + 'SMC': 2, + 'hepatoma': 2, + 'cardiomyocyte/MSCs': 2, + 'FB': 2, + 'eHep': 2, + 'PMHs': 2, + 'Human CM /adult human fibroblasts': 2, + 'HUVSMCs': 2, + 'ACPCs': 2, + 'MC3T3': 2, + 'hNCs': 2, + 'primary rat hepatocyte': 2, + 'Human umbilical vein endothelial cells': 2, + 'HPAAF': 2, + 'VIC': 2, + 'HLC': 2, + 'ADSCs': 2, + 'MFCs': 2, + 'A549 GFP+': 2, + 'ATDC5': 2, + 'CFs/CMs/HUVECs': 2, + 'n neonatal mouse ventricular cardiomyocytes': 2, + 'Human MSCs': 2, + 'Human CPCs / MSCs': 2, + 'hESC-derived HLCs': 2, + 'HCASMCs': 2, + 'iCMs/iCFs/iECs/iCMFs': 1, + 'VICs': 1, + 'iPSCs': 1, + 'primary human hepatocytes': 1, + 'SMCs': 1, + 'iCMs/iCFs/iECs': 1, + 'PC9': 1, + 'H1395': 1, + 'H1650': 1, + 'HUVECs and hiPSC-CS': 1, + 'human neonatal dermal fi broblasts': 1, + '10T1/2 fibroblast-laden cells': 1, + 'iCMFs': 1, + 'neonatal rat ventricular CMs': 1, + 'U87': 1, + 'NSCLC PDX/CAFs': 1, + 'iPSC-derived CM': 1, + 'NIH 3T3 mouse fibroblasts': 1, + 'Neonatal rat cardiomyocytes': 1, + 'hiPSCs-derived HLCs': 1, + 'iPSCs/HUVECs': 1, + 'HULEC-5a': 1, + 'HUVECs and HHSCs': 1, + 'NCI-H1703': 1, + '75% hepatoblast cells, 20% iEC and 5% iMSC': 1, + 'Intrahepatic cholangiocarcinoma (ICC)': 1, + 'NCI-H441': 1, + 'Human Umbilical Vein Endothelial Cells + iPSC-derived CM': 1, + 'rabbit bone marrow mesenchymal stem cells': 1, + 'CMs/CFs': 1, + 'cardiac fibroblasts': 1, + 'iPSCs-derived cardiomyocytes': 1, + 'cardiomyocytes/ fibroblasts': 1, + 'hiPSC-CM': 1, + 'iPSC-CMs': 1, + 'hBM-MSCs': 1, + 'Hepatoblast- single cell/iESC/iMSC': 1, + 'Hepatoblast- spheroid/iESC/iMSC': 1, + 'human primary parathyroid cells': 1, + 'prostate cancer stem cell': 1, + 'HL-1': 1, + 'CPCs / MSCs': 1, + 'pancreatic cancer cell': 1, + 'A549/95-D cells': 1, + 'SMMC-7721': 1, + 'rat ventricular cardiomyocytes': 1, + 'A549 lung adenocarcinoma cells': 1, + 'HMEC-1/ fibroblast/ THP-1': 1, + 'HMEC-1': 1, + 'HAVSMC': 1, + 'HUVEC/HHSC': 1, + 'LX-2': 1, + 'Fibroblasts/THP-1': 1, + 'HLF': 1, + 'HUVECs / LX-2': 1, + 'CCAO': 1, + 'Hepa RG/ HUVEC': 1, + 'BEAS2B': 1, + 'PDGFRΞ²+': 1, + 'hMSCs': 1, + 'CiPSC-HOs': 1, + 'tubular epithelial': 1, +} + +# Density in 10^6 cells/mL, among records using that line. +CELL_DENSITY_RANGES = { + "NoCellCultured": { + "min": 1.0, + "max": 3.0, + "median": 1.0 + }, + "HUVECs": { + "min": 0.035, + "max": 20.0, + "median": 5.0 + }, + "HepG2": { + "min": 1.0, + "max": 20.0, + "median": 1.5 + }, + "bMSCs": { + "min": 0.25, + "max": 20.0, + "median": 10.0 + }, + "chondrocytes": { + "min": 0.1, + "max": 6.0, + "median": 6.0 + }, + "NIH3T3": { + "min": 0.1, + "max": 5.0, + "median": 2.5 + }, + "HepaRG": { + "min": 1.0, + "max": 20.0, + "median": 4.0 + }, + "hiPSC-CMs /ATCCs": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "Myoblast cells": { + "min": 0.366, + "max": 10.0, + "median": 1.0 + }, + "H9c2": { + "min": 0.04, + "max": 1.0, + "median": 1.0 + }, + "HEK": { + "min": 3.0, + "max": 3.0, + "median": 3.0 + }, + "CPCs": { + "min": 2.0, + "max": 8.0, + "median": 2.0 + }, + "MESCs": { + "min": 0.5, + "max": 2.0, + "median": 1.0 + }, + "hiPSCs": { + "min": 1.0, + "max": 15.0, + "median": 15.0 + }, + "L929": { + "min": 0.1, + "max": 10.0, + "median": 1.0 + }, + "Primary mouse hepatocyte": { + "min": 0.128, + "max": 1.0, + "median": 0.128 + }, + "NSCLC PDX": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "human primary kidney cells": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "Cardiac progenitor cells": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "10T1/2": { + "min": 0.5, + "max": 2.0, + "median": 0.5 + }, + "HAVIC": { + "min": 2.5, + "max": 10.0, + "median": 3.75 + }, + "RAMECs": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "hESCs": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "human nasoseptal chondrocytes": { + "min": 4.0, + "max": 4.0, + "median": 4.0 + }, + "PDX": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "PTECs": { + "min": 15.0, + "max": 15.0, + "median": 15.0 + }, + "HPFs": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "MG-63": { + "min": 0.005, + "max": 5.0, + "median": 0.005 + }, + "dermal fibroblasts": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "HASSMC": { + "min": 2.5, + "max": 2.5, + "median": 2.5 + }, + "Saos-2": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "MC3T3-E1": { + "min": 30.0, + "max": 30.0, + "median": 30.0 + }, + "hASCs": { + "min": 1.0, + "max": 2.0, + "median": 1.3 + }, + "SU3": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "ESCs": { + "min": 6.0, + "max": 10.0, + "median": 8.0 + }, + "RPTEC/TERT1": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "hiPSC-CMs and HS-27A": { + "min": 20.0, + "max": 20.0, + "median": 20.0 + }, + "HADMSCs": { + "min": 2.0, + "max": 2.5, + "median": 2.5 + }, + "HACs": { + "min": 5.0, + "max": 15.0, + "median": 5.0 + }, + "MRC5": { + "min": 0.04, + "max": 5.0, + "median": 3.0 + }, + "HeLa": { + "min": 1.0, + "max": 150.0, + "median": 1.0 + }, + "RHECs": { + "min": 0.5, + "max": 0.5, + "median": 0.5 + }, + "cardiomyocytes": { + "min": 0.2, + "max": 10.0, + "median": 6.0 + }, + "human adipogenic mesenchymal stem cells": { + "min": 0.75, + "max": 2.25, + "median": 1.125 + }, + "MSCs": { + "min": 0.2, + "max": 5.0, + "median": 1.0 + }, + "HUVECs/MSCs": { + "min": 6.0, + "max": 6.0, + "median": 6.0 + }, + "U87-MG": { + "min": 5.25, + "max": 5.25, + "median": 5.25 + }, + "RSC96/HUVECs": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "HEPG2/ECs": { + "min": 6.0, + "max": 6.0, + "median": 6.0 + }, + "A549": { + "min": 1.0, + "max": 5.0, + "median": 2.5 + }, + "Human non-small cell lung cancer line Calu-3 (Calu-3)": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "hTMSCs": { + "min": 3.5, + "max": 3.5, + "median": 3.5 + }, + "mouse cardiac cells": { + "min": 0.32, + "max": 0.32, + "median": 0.32 + }, + "Human cardiac fibroblasts": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "Schwann cells": { + "min": 1.0, + "max": 2.0, + "median": 1.0 + }, + "hTERT-MSC": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "bMSCs/hACs": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "EPCs": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "HepaRG / LX-2": { + "min": 1.5, + "max": 3.0, + "median": 3.0 + }, + "Caki-1": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "human induced pluripotent stem cell-derived cardiomyocytes": { + "min": 20.0, + "max": 20.0, + "median": 20.0 + }, + "human epithelial lung carcinoma cells": { + "min": 7.0, + "max": 7.0, + "median": 7.0 + }, + "HCF": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "Huh7": { + "min": 0.5, + "max": 0.5, + "median": 0.5 + }, + "3T3 Fibroblasts": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "EA.hy 926 cells": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "human coronary artery endothelial cells": { + "min": 0.8, + "max": 1.0, + "median": 0.8 + }, + "NIH3T3/ HUVECs": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "neonatal rat ventricular CFs": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "HepG2/C3A": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "IMR-90": { + "min": 3.5, + "max": 3.5, + "median": 3.5 + }, + "rMSC": { + "min": 3.5, + "max": 3.5, + "median": 3.5 + }, + "hiPSC-CM / fibroblasts": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "HUVECs/H9C2": { + "min": 4.0, + "max": 4.0, + "median": 4.0 + }, + "fibroblasts": { + "min": 1.0, + "max": 10.0, + "median": 3.0 + }, + "neonatal rat ventricular cardiomyocytes": { + "min": 1.0, + "max": 200.0, + "median": 1.0 + }, + "Neonatal mouse cardiomyocytes (NMVCMs)": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "murine macrophage-like cell line": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "293FT": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "sADSC": { + "min": 1.0, + "max": 3.0, + "median": 2.0 + }, + "human hepatic stellate cell line": { + "min": 3.0, + "max": 5.0, + "median": 4.0 + }, + "aHSC": { + "min": 3.0, + "max": 5.0, + "median": 4.0 + }, + "NRCMs": { + "min": 20.0, + "max": 20.0, + "median": 20.0 + }, + "HBE": { + "min": 0.13, + "max": 0.13, + "median": 0.13 + }, + "HEK-293": { + "min": 0.05, + "max": 2.0, + "median": 2.0 + }, + "Wnt3a-293FT": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "Human aortic VIC": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "HEK 293FT": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "HNDF": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "Huh7/HepaRG": { + "min": 1.5, + "max": 1.5, + "median": 1.5 + }, + "hiHeps": { + "min": 10.0, + "max": 50.0, + "median": 30.0 + }, + "hCPCs": { + "min": 3.0, + "max": 3.0, + "median": 3.0 + }, + "pancreatic islets cells": { + "min": 0.045, + "max": 0.045, + "median": 0.045 + }, + "ICC": { + "min": 3.0, + "max": 6.0, + "median": 4.5 + }, + "Wnt reporter-293FT": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "Human CPCs": { + "min": 5e-06, + "max": 5.0, + "median": 2.5000025 + }, + "human cardiac progenitor cells": { + "min": 3.0, + "max": 3.0, + "median": 3.0 + }, + "SMC": { + "min": 2.0, + "max": 10.0, + "median": 6.0 + }, + "hepatoma": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "cardiomyocyte/MSCs": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "FB": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "eHep": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "PMHs": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "Human CM /adult human fibroblasts": { + "min": 0.01, + "max": 0.2, + "median": 0.10500000000000001 + }, + "HUVSMCs": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "ACPCs": { + "min": 20.0, + "max": 20.0, + "median": 20.0 + }, + "MC3T3": { + "min": 0.25, + "max": 0.25, + "median": 0.25 + }, + "hNCs": { + "min": 1.0, + "max": 15.0, + "median": 8.0 + }, + "primary rat hepatocyte": { + "min": 1.0, + "max": 5.0, + "median": 3.0 + }, + "Human umbilical vein endothelial cells": { + "min": 1.25, + "max": 10.0, + "median": 5.625 + }, + "HPAAF": { + "min": 0.4, + "max": 4.0, + "median": 2.2 + }, + "VIC": { + "min": 2.0, + "max": 10.0, + "median": 6.0 + }, + "HLC": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "ADSCs": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "MFCs": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "A549 GFP+": { + "min": 1.5, + "max": 1.5, + "median": 1.5 + }, + "ATDC5": { + "min": 5.68, + "max": 5.68, + "median": 5.68 + }, + "CFs/CMs/HUVECs": { + "min": 30.0, + "max": 30.0, + "median": 30.0 + }, + "n neonatal mouse ventricular cardiomyocytes": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "Human MSCs": { + "min": 5e-06, + "max": 5.0, + "median": 2.5000025 + }, + "Human CPCs / MSCs": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "hESC-derived HLCs": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "HCASMCs": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "iCMs/iCFs/iECs/iCMFs": { + "min": 3.87, + "max": 3.87, + "median": 3.87 + }, + "VICs": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "iPSCs": { + "min": 8.0, + "max": 8.0, + "median": 8.0 + }, + "primary human hepatocytes": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "SMCs": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "iCMs/iCFs/iECs": { + "min": 9.6, + "max": 9.6, + "median": 9.6 + }, + "PC9": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "H1395": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "H1650": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "HUVECs and hiPSC-CS": { + "min": 0.01, + "max": 0.01, + "median": 0.01 + }, + "human neonatal dermal fi broblasts": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "10T1/2 fibroblast-laden cells": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "iCMFs": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "neonatal rat ventricular CMs": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "U87": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "NSCLC PDX/CAFs": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "iPSC-derived CM": { + "min": 8.0, + "max": 8.0, + "median": 8.0 + }, + "NIH 3T3 mouse fibroblasts": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "Neonatal rat cardiomyocytes": { + "min": 0.2, + "max": 0.2, + "median": 0.2 + }, + "hiPSCs-derived HLCs": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "iPSCs/HUVECs": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "HULEC-5a": { + "min": 6.0, + "max": 6.0, + "median": 6.0 + }, + "HUVECs and HHSCs": { + "min": 1.8, + "max": 1.8, + "median": 1.8 + }, + "NCI-H1703": { + "min": 6.0, + "max": 6.0, + "median": 6.0 + }, + "75% hepatoblast cells, 20% iEC and 5% iMSC": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "Intrahepatic cholangiocarcinoma (ICC)": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "NCI-H441": { + "min": 6.0, + "max": 6.0, + "median": 6.0 + }, + "Human Umbilical Vein Endothelial Cells + iPSC-derived CM": { + "min": 6.0, + "max": 6.0, + "median": 6.0 + }, + "rabbit bone marrow mesenchymal stem cells": { + "min": 0.1, + "max": 0.1, + "median": 0.1 + }, + "CMs/CFs": { + "min": 30.0, + "max": 30.0, + "median": 30.0 + }, + "cardiac fibroblasts": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "iPSCs-derived cardiomyocytes": { + "min": 200.0, + "max": 200.0, + "median": 200.0 + }, + "cardiomyocytes/ fibroblasts": { + "min": 0.2, + "max": 0.2, + "median": 0.2 + }, + "hiPSC-CM": { + "min": 0.2, + "max": 0.2, + "median": 0.2 + }, + "iPSC-CMs": { + "min": 4.0, + "max": 4.0, + "median": 4.0 + }, + "hBM-MSCs": { + "min": 0.25, + "max": 0.25, + "median": 0.25 + }, + "Hepatoblast- single cell/iESC/iMSC": { + "min": 5.6, + "max": 5.6, + "median": 5.6 + }, + "Hepatoblast- spheroid/iESC/iMSC": { + "min": 5.6, + "max": 5.6, + "median": 5.6 + }, + "human primary parathyroid cells": { + "min": 0.06, + "max": 0.06, + "median": 0.06 + }, + "prostate cancer stem cell": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "HL-1": { + "min": 8.0, + "max": 8.0, + "median": 8.0 + }, + "CPCs / MSCs": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "pancreatic cancer cell": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "A549/95-D cells": { + "min": 1.0, + "max": 1.0, + "median": 1.0 + }, + "SMMC-7721": { + "min": 0.5, + "max": 0.5, + "median": 0.5 + }, + "rat ventricular cardiomyocytes": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "A549 lung adenocarcinoma cells": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "HMEC-1/ fibroblast/ THP-1": { + "min": 35.0, + "max": 35.0, + "median": 35.0 + }, + "HMEC-1": { + "min": 25.0, + "max": 25.0, + "median": 25.0 + }, + "HAVSMC": { + "min": 1.5, + "max": 1.5, + "median": 1.5 + }, + "HUVEC/HHSC": { + "min": 0.9, + "max": 0.9, + "median": 0.9 + }, + "LX-2": { + "min": 0.5, + "max": 0.5, + "median": 0.5 + }, + "Fibroblasts/THP-1": { + "min": 14.25, + "max": 14.25, + "median": 14.25 + }, + "HLF": { + "min": 5.0, + "max": 5.0, + "median": 5.0 + }, + "HUVECs / LX-2": { + "min": 10.0, + "max": 10.0, + "median": 10.0 + }, + "CCAO": { + "min": 0.02, + "max": 0.02, + "median": 0.02 + }, + "Hepa RG/ HUVEC": { + "min": 20.0, + "max": 20.0, + "median": 20.0 + }, + "BEAS2B": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "PDGFR\u03b2+": { + "min": 0.25, + "max": 0.25, + "median": 0.25 + }, + "hMSCs": { + "min": 2.0, + "max": 2.0, + "median": 2.0 + }, + "CiPSC-HOs": { + "min": 2.5, + "max": 2.5, + "median": 2.5 + }, + "tubular epithelial": { + "min": 0.4, + "max": 0.4, + "median": 0.4 + } +} diff --git a/corpus_reference.json b/corpus_reference.json new file mode 100644 index 0000000000000000000000000000000000000000..6ecaefebee6c839471b20999cc328eceb7b67d8c --- /dev/null +++ b/corpus_reference.json @@ -0,0 +1,289 @@ +{ + "biomaterials": [ + "Alginate (%w/v)", + "PVA-HA (%w/v)", + "CaSO4 (%w/v)", + "Na2HPO4 (%w/v)", + "Gelatin (%w/v)", + "GelMA (%w/v)", + "laponite (%w/v)", + "graphene oxide (%w/v)", + "hydroxyapatite (%w/v)", + "Hyaluronic_Acid (%w/v)", + "hyaluronan methacrylate (%w/v)", + "NorHA (%w/v)", + "Fibroin/Fibrinogen (%w/v)", + "Pluronic P-123 (%w/v)", + "Collagen (%w/v)", + "Chitosan (%w/v)", + "CS-AEMA (%w/v)", + "RGD (mM)", + "TCP (%w/v)", + "Gellan (%w/v)", + "bioactive glass (%w/v)", + "Nano/Methylcellulose (%w/v)", + "PEGTA (%w/v)", + "PEGMA (%w/v)", + "PEGDA (%w/v)", + "Agarose (%w/v)", + "hyaluronic acid+ Ph moieties (%w/v)", + "matrigel (%w/v)", + "CaCl2 (mM)", + "NaCl (mM)", + "BaCl2 (mM)", + "SrCl2 (mM)", + "CaCO3 (mM)", + "Genipin (%w/v)", + "PVA (%wt)", + "trans-glutaminase (%w/v)", + "alginate lyase (U/ml)", + "D-glucose (%w/v)", + "PLGA (%w/v)", + "vascular tissue-derived dECM (%w/v)", + "PEG-8-SH (mM)", + "Alginate dialdehyde (%w/v)", + "Alginate sulfate (%w/v)", + "RGD-modified alginate (%w/v)", + "poly(N-isopropylacrylamide) grafted hyaluronan (%w/v)", + "chondroitin sulfate methacrylate (%w/v)", + "PCL (%w/v)", + "alginate methacrylate (%w/v)", + "HRP (U/ml)", + "Pluronic F127 / Lutrol F127 (%w/v)", + "Irgacure 2959 (%w/v)", + "Eosin Y (%w/v)", + "Ruthenium (mM)", + "sodium persulfate (SPS) (mM)", + "HEPES (mM)", + "LAP (%w/v)", + "glutaraldehyde (%w/v)", + "PBS (M)", + "glycerol (%w/v)", + "cECM (%w/v)", + "gel-fu (%w/v)", + "Rose Bengal (%w/v)", + "Vitamin B2 (%w/v)", + "VEGF (%w/v)", + "Polypyrrole:PSS (%w/v)", + "borate bioactive glass (%w/v)", + "astaxanthin (%w/v)", + "PRP (%v/v)", + "methacrylated collagen (%w/v)", + "\u03b1-Toc (\u00b5M)", + "ascorbic acid (mM)", + "Liver dECM (%w/v)", + "galactosylated alginate (%w/v)", + "SC-PEG (%w/v)", + "SFMA-L (%w/v)", + "SFMA-M (%w/v)", + "SFMA-H (%w/v)", + "KdECMMA (%w/v)", + "BA silk fibroin (%w/v)", + "Carrageenan (%v)", + "Carbopol ETD 2020 NF (%w/v)", + "Carbopol Ultrez 10 NF (%w/v)", + "Carbopol NF-980 (%w/v)", + "FBS (%v/v)", + "MeTro (%w/v)", + "Triethanolamine (%v/v)", + "PEG-Fibrinogen (%w/v)", + "polyethylene glycol dimethacrylate (%w/v)", + "aprotinin (\u00b5g/ml)", + "gold nanorod (mg/mL)", + "egg white (w/v)", + "1-Vinyl-2-pyrrolidone (v/v)", + "carboxyl functionalized carbon nanotubes (%w/v)", + "polyHIPE (%w/v)", + "\u03b2-D galactose (mM)", + "hydrogen peroxide (H2O2) (%v/v)", + "lactic acid (v/v)", + "NorCol (%w/v)", + "DTT (%w/v)", + "ammonium persulfate (mM)", + "diTyr-RGD (mM)", + "PHEG-Tyr (%w/v)", + "MMP2-degradable peptide (%w/v)", + "KdECM (%w/v)", + "EDC (mg)", + "NHS (mg)", + "VA086 (%w/v)", + "PGS (%w/v)", + "thiolated HA (%w/v)", + "boron nitride nanotubes (%w/v)", + "PEDOT:PSS (ul)", + "KCl (mM)", + "skeletal muscle ECM methacrylate (%w/v)", + "PEO (%w/v)", + "Carbon dots (mg/ml)", + "Laminin (ug/ml)", + "DF-PEG (%w/v)", + "omentum ECM (%w/v)", + "thrombin (unit/ml)", + "Carbon nanotube (CNT) (w/v)", + "Phytagel (%v)", + "Laponite-XLG (%w/w)", + "sodium carboxymethyl cellulose (mg)", + "lysozyme amyloid nanofibrils:gold nanoparticles (mg)", + "Lysozyme amyloid nanofibrils (mg)", + "rGO (mg/ml)", + "Methacrylated gellan gum (%w/v)", + "Acetylsalicylic Acid (%w/w)", + "PVA methacrylate (%w/v)", + "MXene (mg/ml)" + ], + "print_params": [ + "Physical Crosslinking Duration (s)", + "Photo Crosslinking Duration (s)", + "Extrusion Pressure (kPa)", + "Nozzle Movement Speed (mm/s)", + "Nozzle Diameter (\u00b5m)", + "Syringe Temperature (\u00b0C)", + "Substrate Temperature (\u00b0C)" + ], + "predictors": [ + "Alginate (%w/v)", + "PVA-HA (%w/v)", + "CaSO4 (%w/v)", + "Na2HPO4 (%w/v)", + "Gelatin (%w/v)", + "GelMA (%w/v)", + "laponite (%w/v)", + "graphene oxide (%w/v)", + "hydroxyapatite (%w/v)", + "Hyaluronic_Acid (%w/v)", + "hyaluronan methacrylate (%w/v)", + "NorHA (%w/v)", + "Fibroin/Fibrinogen (%w/v)", + "Pluronic P-123 (%w/v)", + "Collagen (%w/v)", + "Chitosan (%w/v)", + "CS-AEMA (%w/v)", + "RGD (mM)", + "TCP (%w/v)", + "Gellan (%w/v)", + "bioactive glass (%w/v)", + "Nano/Methylcellulose (%w/v)", + "PEGTA (%w/v)", + "PEGMA (%w/v)", + "PEGDA (%w/v)", + "Agarose (%w/v)", + "hyaluronic acid+ Ph moieties (%w/v)", + "matrigel (%w/v)", + "CaCl2 (mM)", + "NaCl (mM)", + "BaCl2 (mM)", + "SrCl2 (mM)", + "CaCO3 (mM)", + "Genipin (%w/v)", + "PVA (%wt)", + "trans-glutaminase (%w/v)", + "alginate lyase (U/ml)", + "D-glucose (%w/v)", + "PLGA (%w/v)", + "vascular tissue-derived dECM (%w/v)", + "PEG-8-SH (mM)", + "Alginate dialdehyde (%w/v)", + "Alginate sulfate (%w/v)", + "RGD-modified alginate (%w/v)", + "poly(N-isopropylacrylamide) grafted hyaluronan (%w/v)", + "chondroitin sulfate methacrylate (%w/v)", + "PCL (%w/v)", + "alginate methacrylate (%w/v)", + "HRP (U/ml)", + "Pluronic F127 / Lutrol F127 (%w/v)", + "Irgacure 2959 (%w/v)", + "Eosin Y (%w/v)", + "Ruthenium (mM)", + "sodium persulfate (SPS) (mM)", + "HEPES (mM)", + "LAP (%w/v)", + "glutaraldehyde (%w/v)", + "PBS (M)", + "glycerol (%w/v)", + "cECM (%w/v)", + "gel-fu (%w/v)", + "Rose Bengal (%w/v)", + "Vitamin B2 (%w/v)", + "VEGF (%w/v)", + "Polypyrrole:PSS (%w/v)", + "borate bioactive glass (%w/v)", + "astaxanthin (%w/v)", + "PRP (%v/v)", + "methacrylated collagen (%w/v)", + "\u03b1-Toc (\u00b5M)", + "ascorbic acid (mM)", + "Liver dECM (%w/v)", + "galactosylated alginate (%w/v)", + "SC-PEG (%w/v)", + "SFMA-L (%w/v)", + "SFMA-M (%w/v)", + "SFMA-H (%w/v)", + "KdECMMA (%w/v)", + "BA silk fibroin (%w/v)", + "Carrageenan (%v)", + "Carbopol ETD 2020 NF (%w/v)", + "Carbopol Ultrez 10 NF (%w/v)", + "Carbopol NF-980 (%w/v)", + "FBS (%v/v)", + "MeTro (%w/v)", + "Triethanolamine (%v/v)", + "PEG-Fibrinogen (%w/v)", + "polyethylene glycol dimethacrylate (%w/v)", + "aprotinin (\u00b5g/ml)", + "gold nanorod (mg/mL)", + "egg white (w/v)", + "1-Vinyl-2-pyrrolidone (v/v)", + "carboxyl functionalized carbon nanotubes (%w/v)", + "polyHIPE (%w/v)", + "\u03b2-D galactose (mM)", + "hydrogen peroxide (H2O2) (%v/v)", + "lactic acid (v/v)", + "NorCol (%w/v)", + "DTT (%w/v)", + "ammonium persulfate (mM)", + "diTyr-RGD (mM)", + "PHEG-Tyr (%w/v)", + "MMP2-degradable peptide (%w/v)", + "KdECM (%w/v)", + "EDC (mg)", + "NHS (mg)", + "VA086 (%w/v)", + "PGS (%w/v)", + "thiolated HA (%w/v)", + "boron nitride nanotubes (%w/v)", + "PEDOT:PSS (ul)", + "KCl (mM)", + "skeletal muscle ECM methacrylate (%w/v)", + "PEO (%w/v)", + "Carbon dots (mg/ml)", + "Laminin (ug/ml)", + "DF-PEG (%w/v)", + "omentum ECM (%w/v)", + "thrombin (unit/ml)", + "Carbon nanotube (CNT) (w/v)", + "Phytagel (%v)", + "Laponite-XLG (%w/w)", + "sodium carboxymethyl cellulose (mg)", + "lysozyme amyloid nanofibrils:gold nanoparticles (mg)", + "Lysozyme amyloid nanofibrils (mg)", + "rGO (mg/ml)", + "Methacrylated gellan gum (%w/v)", + "Acetylsalicylic Acid (%w/w)", + "PVA methacrylate (%w/v)", + "MXene (mg/ml)", + "Cell Line", + "Cell Density (million cells/mL)", + "Physical Crosslinking Duration (s)", + "Photo Crosslinking Duration (s)", + "Extrusion Pressure (kPa)", + "Nozzle Movement Speed (mm/s)", + "Nozzle Diameter (\u00b5m)", + "Syringe Temperature (\u00b0C)", + "Substrate Temperature (\u00b0C)" + ], + "targets": [ + "Printability", + "Cell Response" + ], + "n_records": 2646 +} \ No newline at end of file diff --git a/corpus_reference.parquet b/corpus_reference.parquet new file mode 100644 index 0000000000000000000000000000000000000000..d1b1c2bf93e43db369289b1f31e6e5b0b43d8134 --- /dev/null +++ b/corpus_reference.parquet @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:93af4ac5ab671ead33c91de0ebcc7888b08c1d8b41abb217445f5404a609d584 +size 116700 diff --git a/deploy/models/classifiers/random/cell_response__adaboost.pkl b/deploy/models/classifiers/random/cell_response__adaboost.pkl new file mode 100644 index 0000000000000000000000000000000000000000..017caef1c356bda88c7fce7a56968b7defa7ba1d --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__adaboost.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8933115563a17d4ba1ce2bcb15daa1564c86e5b697a11a3350c8424fef507f93 +size 44341 diff --git a/deploy/models/classifiers/random/cell_response__bagged_trees.pkl b/deploy/models/classifiers/random/cell_response__bagged_trees.pkl new file mode 100644 index 0000000000000000000000000000000000000000..8b734c4e0afa001d65026cf81c22cdc3bf65fad2 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__bagged_trees.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ec7fd79460aca6c9212d7757da8b19b01f50ef17b35fbb1e4817a935429fd8f3 +size 1306142 diff --git a/deploy/models/classifiers/random/cell_response__bernoulli_naive_bayes.pkl b/deploy/models/classifiers/random/cell_response__bernoulli_naive_bayes.pkl new file mode 100644 index 0000000000000000000000000000000000000000..cc3a48e886d6cb17228c64b5ba19d8a017710868 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__bernoulli_naive_bayes.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:282feda9b6e91e96d5dd1ec9cf6ec43d51e8658a38aae0ff94f763ee9e800368 +size 4218 diff --git a/deploy/models/classifiers/random/cell_response__catboost.pkl b/deploy/models/classifiers/random/cell_response__catboost.pkl new file mode 100644 index 0000000000000000000000000000000000000000..dc7f3b2a22bd5589d741fe48334516c821a8d528 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__catboost.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e8acf6b8932614de2bd91250155ade47fe8a9249d4190d24aacb7212f8ec9a9f +size 110986 diff --git a/deploy/models/classifiers/random/cell_response__decision_tree.pkl b/deploy/models/classifiers/random/cell_response__decision_tree.pkl new file mode 100644 index 0000000000000000000000000000000000000000..047c494c72650bbadf9f2f466eecdd9405be2dfd --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__decision_tree.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fec56bf6dad067b23827be89f8a816e5c10871268acce82da80894fbb58a5e25 +size 3478 diff --git a/deploy/models/classifiers/random/cell_response__extra_tree.pkl b/deploy/models/classifiers/random/cell_response__extra_tree.pkl new file mode 100644 index 0000000000000000000000000000000000000000..8e48a8fa60b24aca9c1cbed0f106ab34ad747b2c --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__extra_tree.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0128ac01ef824a6b8df6f745106d695b6a593a292666ae517d3d68924d514153 +size 3469 diff --git a/deploy/models/classifiers/random/cell_response__extra_trees.pkl b/deploy/models/classifiers/random/cell_response__extra_trees.pkl new file mode 100644 index 0000000000000000000000000000000000000000..75c8b463045e8d230620b04b6f34f1cd159c743b --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__extra_trees.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2a1ebaf96b9d13fcc91b5a3c1ca09633eab16f7b33162624917e435cde2f2efd +size 1600493 diff --git a/deploy/models/classifiers/random/cell_response__gaussian_naive_bayes.pkl b/deploy/models/classifiers/random/cell_response__gaussian_naive_bayes.pkl new file mode 100644 index 0000000000000000000000000000000000000000..7f259ccf8fe3140966d1cb3593e951bb30692e5a --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__gaussian_naive_bayes.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d00e8cc7debd29d176894f559c3b043bd0ef03c6b1d0ecc6c86fa2149f6855d0 +size 7337 diff --git a/deploy/models/classifiers/random/cell_response__gradient_boosting.pkl b/deploy/models/classifiers/random/cell_response__gradient_boosting.pkl new file mode 100644 index 0000000000000000000000000000000000000000..b34d78f59232f32bc17a7e2b4d847a8f7a950db1 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__gradient_boosting.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0a4332f43c8af0932c05662b97fde1f0d67525773457cd72ddcc670273462767 +size 28636 diff --git a/deploy/models/classifiers/random/cell_response__hist_gradient_boosting.pkl b/deploy/models/classifiers/random/cell_response__hist_gradient_boosting.pkl new file mode 100644 index 0000000000000000000000000000000000000000..8692468053a620234949c2f5a5f27f7e545968e9 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__hist_gradient_boosting.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2bf42d2ad81b2b130b6d295e9d97480d9ada674e89d701575c2bf7be6d379ecb +size 645934 diff --git a/deploy/models/classifiers/random/cell_response__k_nearest_neighbours.pkl b/deploy/models/classifiers/random/cell_response__k_nearest_neighbours.pkl new file mode 100644 index 0000000000000000000000000000000000000000..20c175eccfb494e0cf3aecb3c5f5f52711b1f5f1 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__k_nearest_neighbours.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:780fbe925315a38f89bbf907dac7e4b1f587266533df7c0115e0fc04276b40a0 +size 53470 diff --git a/deploy/models/classifiers/random/cell_response__k_nn_distance_weighted.pkl b/deploy/models/classifiers/random/cell_response__k_nn_distance_weighted.pkl new file mode 100644 index 0000000000000000000000000000000000000000..4add6c6a102cb33066a1d889d9640718451f6121 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__k_nn_distance_weighted.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:534a360ee86a358052a823d1107c6c002e4e9ef58a3e84f1c2c860f025c2222d +size 127106 diff --git a/deploy/models/classifiers/random/cell_response__lightgbm.pkl b/deploy/models/classifiers/random/cell_response__lightgbm.pkl new file mode 100644 index 0000000000000000000000000000000000000000..78d1cbf9bb737d9cbb1c04595d560af0a33b472b --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__lightgbm.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2c198a22bf0a01258f0e9196472c74797ae0529ab244b1c3e0fe0d6c40e1d23a +size 552411 diff --git a/deploy/models/classifiers/random/cell_response__linear_discriminant.pkl b/deploy/models/classifiers/random/cell_response__linear_discriminant.pkl new file mode 100644 index 0000000000000000000000000000000000000000..4dd13d52772f9f6fc2a833a07394f52c6c755048 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__linear_discriminant.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:14d6b11c6b801eef3e7f9fa0c6ad28c0a9366fad15cce208bd00725d313adcb2 +size 145025 diff --git a/deploy/models/classifiers/random/cell_response__linear_svm.pkl b/deploy/models/classifiers/random/cell_response__linear_svm.pkl new file mode 100644 index 0000000000000000000000000000000000000000..2a43a9f98d77676dc67945e517c4da357e9dc9e9 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__linear_svm.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f41c3e869a8250cbbf74ea42271cd3a07684d46794b48a1ff042f29cf8cd9d9 +size 5535 diff --git a/deploy/models/classifiers/random/cell_response__logistic_regression.pkl b/deploy/models/classifiers/random/cell_response__logistic_regression.pkl new file mode 100644 index 0000000000000000000000000000000000000000..bc5d45247c5478daa519c7cdb39e212cb06edd1e --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__logistic_regression.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fe67a6655375a6e0bfcc706115cc992ed566ba87e57a5696f22843b1b681c1a9 +size 6535 diff --git a/deploy/models/classifiers/random/cell_response__logistic_regression_balanced.pkl b/deploy/models/classifiers/random/cell_response__logistic_regression_balanced.pkl new file mode 100644 index 0000000000000000000000000000000000000000..571e087e7df6300280e95ddd89c0ef58ceba2d2d --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__logistic_regression_balanced.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc90e1018ef9d648de7adb5da1b60e267f4b1e357f538b8fdaa5c6b246c0c97d +size 6514 diff --git a/deploy/models/classifiers/random/cell_response__mlp_256_128.pkl b/deploy/models/classifiers/random/cell_response__mlp_256_128.pkl new file mode 100644 index 0000000000000000000000000000000000000000..e37a716d6f3d478396212b700dc8113772e0a1f2 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__mlp_256_128.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2f4bf03e58aea3bb7c171b22d4bf03077652236328dcbf30317765bd28c95a35 +size 877150 diff --git a/deploy/models/classifiers/random/cell_response__nearest_centroid.pkl b/deploy/models/classifiers/random/cell_response__nearest_centroid.pkl new file mode 100644 index 0000000000000000000000000000000000000000..43cf0d5bd2194e0fb3b0463be402fba69babd01a --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__nearest_centroid.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:579188d8f4730741cc6d4fcec2d042de26c5f1c4e9810ab23851e2152ec467e3 +size 7171 diff --git a/deploy/models/classifiers/random/cell_response__passive_aggressive.pkl b/deploy/models/classifiers/random/cell_response__passive_aggressive.pkl new file mode 100644 index 0000000000000000000000000000000000000000..c596abb82ab0a31dfd20ebac729a184eac966c14 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__passive_aggressive.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6d8c577687544c0044204267fce8e94a8fcd695be0e7d8e4660e029bf2b409f7 +size 12296 diff --git a/deploy/models/classifiers/random/cell_response__perceptron.pkl b/deploy/models/classifiers/random/cell_response__perceptron.pkl new file mode 100644 index 0000000000000000000000000000000000000000..04e831d2a71c8f1fa76e9ddce3534acc413f8dde --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__perceptron.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:12bd23ecb4a5e58d8a3ba47a549bc38b7e9d027a1dde92c474a5e673b8cd3d34 +size 4111 diff --git a/deploy/models/classifiers/random/cell_response__quadratic_discriminant.pkl b/deploy/models/classifiers/random/cell_response__quadratic_discriminant.pkl new file mode 100644 index 0000000000000000000000000000000000000000..bfd375ce878d2a8c0d1edde0cf1f1510e0459bf4 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__quadratic_discriminant.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:006ecba15ba6131d1dff15163c6168615363be3716e8eb3fc96a4b63b0f44fc7 +size 29521 diff --git a/deploy/models/classifiers/random/cell_response__random_forest.pkl b/deploy/models/classifiers/random/cell_response__random_forest.pkl new file mode 100644 index 0000000000000000000000000000000000000000..30cfee0993cc690843089867349a814ec04b08c0 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__random_forest.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1b5b4c7ddd69ecf0575e2cad3dc3b94992ded2934f1fbc74498182984e396165 +size 4994 diff --git a/deploy/models/classifiers/random/cell_response__random_forest_balanced.pkl b/deploy/models/classifiers/random/cell_response__random_forest_balanced.pkl new file mode 100644 index 0000000000000000000000000000000000000000..bd532ae44a58103be95b677baf8118917e06fac7 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__random_forest_balanced.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7be9e347e9d4921977931954bf9d2326544452d96d7bcd2e9ddbfae163a95b77 +size 3902841 diff --git a/deploy/models/classifiers/random/cell_response__ridge_classifier.pkl b/deploy/models/classifiers/random/cell_response__ridge_classifier.pkl new file mode 100644 index 0000000000000000000000000000000000000000..06c87b9e0b30df2bd9c349cc8f74060f42c5b1a8 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__ridge_classifier.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:887429d6d2f49ef7c671d9be75c29bb94bc1c7e3abbe202dba13596533c96af9 +size 6651 diff --git a/deploy/models/classifiers/random/cell_response__sgd_hinge.pkl b/deploy/models/classifiers/random/cell_response__sgd_hinge.pkl new file mode 100644 index 0000000000000000000000000000000000000000..ea50fd72afe7ec2674c668fc9c0b07929ecbadff --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__sgd_hinge.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0070b38894cb7ef91f72dae63d7efe23ce835ee369e481d3b366062c886d3baa +size 4886 diff --git a/deploy/models/classifiers/random/cell_response__soft_voting_rf_xgb_lr.pkl b/deploy/models/classifiers/random/cell_response__soft_voting_rf_xgb_lr.pkl new file mode 100644 index 0000000000000000000000000000000000000000..79bf5d65fcd989401e4812a76a3753a9c0ec97c4 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__soft_voting_rf_xgb_lr.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19f3ea953fb061519b7d0fa42416c8edacffc93f762e87cab7a90f5692643340 +size 5233168 diff --git a/deploy/models/classifiers/random/cell_response__stacking_rf_xgb_lr_lr.pkl b/deploy/models/classifiers/random/cell_response__stacking_rf_xgb_lr_lr.pkl new file mode 100644 index 0000000000000000000000000000000000000000..22c788506cb761efed15a90f5a3c91b78bd1ba67 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__stacking_rf_xgb_lr_lr.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6ed798ca8edf3a2f54aae8fde0cc0aaa3eba4457897b97ffa19be49c85c035ba +size 5233839 diff --git a/deploy/models/classifiers/random/cell_response__svm_polynomial.pkl b/deploy/models/classifiers/random/cell_response__svm_polynomial.pkl new file mode 100644 index 0000000000000000000000000000000000000000..50aa98631c91f5633cd2a86e131cdb704a5a3883 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__svm_polynomial.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd644c182976274d6ec346169b332e3f18d29912c2152e91f3b6ae17c3155a34 +size 60088 diff --git a/deploy/models/classifiers/random/cell_response__svm_rbf.pkl b/deploy/models/classifiers/random/cell_response__svm_rbf.pkl new file mode 100644 index 0000000000000000000000000000000000000000..2204db0e8c7431af9ee262581aebd59cc69e47f8 --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__svm_rbf.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ec357aa39dbb6631b933c1e98e4d4b2df6135cc5237a40b2d24dfb6c7d128e45 +size 66004 diff --git a/deploy/models/classifiers/random/cell_response__xgboost.pkl b/deploy/models/classifiers/random/cell_response__xgboost.pkl new file mode 100644 index 0000000000000000000000000000000000000000..aa3181e7f925c6170fbe4c7a39321c7145e7172b --- /dev/null +++ b/deploy/models/classifiers/random/cell_response__xgboost.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e9408add167a4c715a875fd88daf4457c46f058ba91c8fa6f9ca57ce1b7d9bb8 +size 189666 diff --git a/deploy/models/classifiers/random/printability__adaboost.pkl b/deploy/models/classifiers/random/printability__adaboost.pkl new file mode 100644 index 0000000000000000000000000000000000000000..c8cb37979e85e6008e2ff50cd1c96eadd1f863a2 --- /dev/null +++ b/deploy/models/classifiers/random/printability__adaboost.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:61fa142eee65a9a3dab6e40de5ec7715340c56563254ca52623e45f0a3681625 +size 48212 diff --git a/deploy/models/classifiers/random/printability__bagged_trees.pkl b/deploy/models/classifiers/random/printability__bagged_trees.pkl new file mode 100644 index 0000000000000000000000000000000000000000..1b21ed4c8110e5ebac1fde706b617a318d8c775a --- /dev/null +++ b/deploy/models/classifiers/random/printability__bagged_trees.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:98d12f997b7e7bcb6789ddbf32dacddcc53c859e265778f18682b7b693397e51 +size 3775738 diff --git a/deploy/models/classifiers/random/printability__bernoulli_naive_bayes.pkl b/deploy/models/classifiers/random/printability__bernoulli_naive_bayes.pkl new file mode 100644 index 0000000000000000000000000000000000000000..ad30d22d3f8c2df554e94d15098e07e61f00b35a --- /dev/null +++ b/deploy/models/classifiers/random/printability__bernoulli_naive_bayes.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:98d82bc7b87ea7a6e51932768636f4f9cb2e62d7f796d1f1f2aa420230ae0071 +size 3497 diff --git a/deploy/models/classifiers/random/printability__catboost.pkl b/deploy/models/classifiers/random/printability__catboost.pkl new file mode 100644 index 0000000000000000000000000000000000000000..c7973120687f66f1b5f4d43d95d9db335dd191dd --- /dev/null +++ b/deploy/models/classifiers/random/printability__catboost.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:45c920875ad42ed56cade9561ab095e950e209819d68f2e061ff68534dd73fd3 +size 787008 diff --git a/deploy/models/classifiers/random/printability__decision_tree.pkl b/deploy/models/classifiers/random/printability__decision_tree.pkl new file mode 100644 index 0000000000000000000000000000000000000000..d1c15221e37eb85d2c717ced8200e4f7b978a667 --- /dev/null +++ b/deploy/models/classifiers/random/printability__decision_tree.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:768ec1872e1fc323154af785e6f71d554389de56fb43aa2c0624e6a70852a6b5 +size 10029 diff --git a/deploy/models/classifiers/random/printability__extra_tree.pkl b/deploy/models/classifiers/random/printability__extra_tree.pkl new file mode 100644 index 0000000000000000000000000000000000000000..a2c5a21a9f3c387a3027cf83b23bb9ca422a45e8 --- /dev/null +++ b/deploy/models/classifiers/random/printability__extra_tree.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:591f152e86031f04d9d92a5cd88164dcb636509e63d6726010c6814b512bfa75 +size 9601 diff --git a/deploy/models/classifiers/random/printability__extra_trees.pkl b/deploy/models/classifiers/random/printability__extra_trees.pkl new file mode 100644 index 0000000000000000000000000000000000000000..f24752d40524d82d373189cac889d9123eba24e1 --- /dev/null +++ b/deploy/models/classifiers/random/printability__extra_trees.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e152e1e7423d5e3306b7abe26ee13ed67596433b1e910f818ed6840a284fffa7 +size 487961 diff --git a/deploy/models/classifiers/random/printability__gaussian_naive_bayes.pkl b/deploy/models/classifiers/random/printability__gaussian_naive_bayes.pkl new file mode 100644 index 0000000000000000000000000000000000000000..960f73feb785386dbbd466ae1bff02e4c0b6ffc7 --- /dev/null +++ b/deploy/models/classifiers/random/printability__gaussian_naive_bayes.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d806b6407b0cc8ab12d7fa836dee8811e349c8e740b7983016111b02d2239223 +size 6142 diff --git a/deploy/models/classifiers/random/printability__gradient_boosting.pkl b/deploy/models/classifiers/random/printability__gradient_boosting.pkl new file mode 100644 index 0000000000000000000000000000000000000000..033d09710e14421fa052fc9d6b20727664cbda3b --- /dev/null +++ b/deploy/models/classifiers/random/printability__gradient_boosting.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:47a1931819018e107c00db7279b2e46722b92606c7d7fc1bf95f5f2728e9e7fa +size 955239 diff --git a/deploy/models/classifiers/random/printability__hist_gradient_boosting.pkl b/deploy/models/classifiers/random/printability__hist_gradient_boosting.pkl new file mode 100644 index 0000000000000000000000000000000000000000..8a389d8dc652ca107eca536153b3f9b7c5fbb3fd --- /dev/null +++ b/deploy/models/classifiers/random/printability__hist_gradient_boosting.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8503f4f620cc539db9e017106b8df556958c3e3191c60da4568d428ac0e2f64f +size 1555799 diff --git a/deploy/models/classifiers/random/printability__k_nearest_neighbours.pkl b/deploy/models/classifiers/random/printability__k_nearest_neighbours.pkl new file mode 100644 index 0000000000000000000000000000000000000000..e794e7a5260383fea518f516034055bb3f98436e --- /dev/null +++ b/deploy/models/classifiers/random/printability__k_nearest_neighbours.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:454d01feb6c9b0927614bf2537641b445aac81664bb0df17b33a7472fccceb49 +size 53695 diff --git a/deploy/models/classifiers/random/printability__k_nn_distance_weighted.pkl b/deploy/models/classifiers/random/printability__k_nn_distance_weighted.pkl new file mode 100644 index 0000000000000000000000000000000000000000..547d8bd15d0bef00e070d8c1699c5c516f2c8105 --- /dev/null +++ b/deploy/models/classifiers/random/printability__k_nn_distance_weighted.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:09f59127db7f4e8d30b7ad810bcdd3eda19749fb13b10ac68fe4e14a32991767 +size 53689 diff --git a/deploy/models/classifiers/random/printability__lightgbm.pkl b/deploy/models/classifiers/random/printability__lightgbm.pkl new file mode 100644 index 0000000000000000000000000000000000000000..489ae76f1a9709081de0183c17aafc321d913efc --- /dev/null +++ b/deploy/models/classifiers/random/printability__lightgbm.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59b18f86de402feafc810534cd093c2fe77af87772a1a20c37d050fbc643a1cc +size 1783453 diff --git a/deploy/models/classifiers/random/printability__linear_discriminant.pkl b/deploy/models/classifiers/random/printability__linear_discriminant.pkl new file mode 100644 index 0000000000000000000000000000000000000000..835d66457bcd7b510376ecdf2909d1192cf031dc --- /dev/null +++ b/deploy/models/classifiers/random/printability__linear_discriminant.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7fc2a8a5bf25dc66cabbcc83dc091e8e36d9f7457ca09595f0ca7abc8086342f +size 142568 diff --git a/deploy/models/classifiers/random/printability__linear_svm.pkl b/deploy/models/classifiers/random/printability__linear_svm.pkl new file mode 100644 index 0000000000000000000000000000000000000000..4c5380c7433ee83d224cf82dc56e56047a28ed5d --- /dev/null +++ b/deploy/models/classifiers/random/printability__linear_svm.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5003a237f95531955627e47a8b231ebc28e5ac24a3e4a9ba7d6f36b63d2ee226 +size 5296 diff --git a/deploy/models/classifiers/random/printability__logistic_regression.pkl b/deploy/models/classifiers/random/printability__logistic_regression.pkl new file mode 100644 index 0000000000000000000000000000000000000000..7d06a2f879980a241479e20781c4a989413bceff --- /dev/null +++ b/deploy/models/classifiers/random/printability__logistic_regression.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e997dcc201a4019f483df41d3c34b500e5ddc28a3c07fa6127a484128f49ce14 +size 5402 diff --git a/deploy/models/classifiers/random/printability__logistic_regression_balanced.pkl b/deploy/models/classifiers/random/printability__logistic_regression_balanced.pkl new file mode 100644 index 0000000000000000000000000000000000000000..fb9089c0e9f6fc312a22391f2d759ecec77aa044 --- /dev/null +++ b/deploy/models/classifiers/random/printability__logistic_regression_balanced.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f60a3bf8190a0775b7e2cea7a65e037e9ead9001675af5ff2ed8c10e672749f +size 5487 diff --git a/deploy/models/classifiers/random/printability__mlp_256_128.pkl b/deploy/models/classifiers/random/printability__mlp_256_128.pkl new file mode 100644 index 0000000000000000000000000000000000000000..d2a9d1c9174be474a4ac6cfddf1646c46e68a87f --- /dev/null +++ b/deploy/models/classifiers/random/printability__mlp_256_128.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a8aa16d753eb9aa0e2cb19177aec37ed56c01e9e714fd41977a4a9ddc303c892 +size 2479623 diff --git a/deploy/models/classifiers/random/printability__nearest_centroid.pkl b/deploy/models/classifiers/random/printability__nearest_centroid.pkl new file mode 100644 index 0000000000000000000000000000000000000000..360ebbd96e135d5b13514d7c9dbf3609d0aaa841 --- /dev/null +++ b/deploy/models/classifiers/random/printability__nearest_centroid.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3168f56946288cec73141c20b257408f14c852bd82c626b3387d35427b76297e +size 10675 diff --git a/deploy/models/classifiers/random/printability__passive_aggressive.pkl b/deploy/models/classifiers/random/printability__passive_aggressive.pkl new file mode 100644 index 0000000000000000000000000000000000000000..8271e51050310e49ec0582bd48297504d81de8dd --- /dev/null +++ b/deploy/models/classifiers/random/printability__passive_aggressive.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6be8bfb9c6800ce38759f3d005afa519170f8cb4aa0f32239f1e9c42e5b29bc8 +size 10140 diff --git a/deploy/models/classifiers/random/printability__perceptron.pkl b/deploy/models/classifiers/random/printability__perceptron.pkl new file mode 100644 index 0000000000000000000000000000000000000000..7ca96243f348108e0cd855a69f0ccd35c0e5ecef --- /dev/null +++ b/deploy/models/classifiers/random/printability__perceptron.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:495b685690763c2aab0212fa584ce4e145709cf8a44479b760a320f1e13a8f25 +size 3788 diff --git a/deploy/models/classifiers/random/printability__quadratic_discriminant.pkl b/deploy/models/classifiers/random/printability__quadratic_discriminant.pkl new file mode 100644 index 0000000000000000000000000000000000000000..20536b6ba7594d8869d2994d5d9fce8b35750db0 --- /dev/null +++ b/deploy/models/classifiers/random/printability__quadratic_discriminant.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7424e80ca7b551781302e507fe091b26669c442d5fb984ae60b456cd23d50f6b +size 140334 diff --git a/deploy/models/classifiers/random/printability__random_forest.pkl b/deploy/models/classifiers/random/printability__random_forest.pkl new file mode 100644 index 0000000000000000000000000000000000000000..eff3b241c7996bdec01acc98ad44981cbfaa3867 --- /dev/null +++ b/deploy/models/classifiers/random/printability__random_forest.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:733cbf8260af6d48d37e4957131e63a12c3f97a0f8af09d7dd4671c27084bbd3 +size 262948 diff --git a/deploy/models/classifiers/random/printability__random_forest_balanced.pkl b/deploy/models/classifiers/random/printability__random_forest_balanced.pkl new file mode 100644 index 0000000000000000000000000000000000000000..0213c8494901edbd3d10c3250b55a90d7901a994 --- /dev/null +++ b/deploy/models/classifiers/random/printability__random_forest_balanced.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1d3ca33c2d3fa2f80dd020a52e2f9aa734426a255039ddbeb26978294b1b056d +size 2364519 diff --git a/deploy/models/classifiers/random/printability__ridge_classifier.pkl b/deploy/models/classifiers/random/printability__ridge_classifier.pkl new file mode 100644 index 0000000000000000000000000000000000000000..747dc7a996558c6099c0dc25b61ff3c8d20d4902 --- /dev/null +++ b/deploy/models/classifiers/random/printability__ridge_classifier.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3ec918b7a29e97eb50805c91628d368fdeb6c69fe8e8e67e22ef11b4e9ec072b +size 5460 diff --git a/deploy/models/classifiers/random/printability__sgd_hinge.pkl b/deploy/models/classifiers/random/printability__sgd_hinge.pkl new file mode 100644 index 0000000000000000000000000000000000000000..946021b5e5f6d8e1b1ff5152eae0a15382fceb26 --- /dev/null +++ b/deploy/models/classifiers/random/printability__sgd_hinge.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:52d0ab3869aab543e325f80253801b31222adda8bf462cad90f81aff6c9a5a62 +size 5438 diff --git a/deploy/models/classifiers/random/printability__soft_voting_rf_xgb_lr.pkl b/deploy/models/classifiers/random/printability__soft_voting_rf_xgb_lr.pkl new file mode 100644 index 0000000000000000000000000000000000000000..dcc52e6931f3fc89904b0e5d3fce7d16cf18d1c6 --- /dev/null +++ b/deploy/models/classifiers/random/printability__soft_voting_rf_xgb_lr.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:157f10b8830b617c7ceaddde35d625713c2dd4e0dfef8066a2ebfde00560f578 +size 6042980 diff --git a/deploy/models/classifiers/random/printability__stacking_rf_xgb_lr_lr.pkl b/deploy/models/classifiers/random/printability__stacking_rf_xgb_lr_lr.pkl new file mode 100644 index 0000000000000000000000000000000000000000..04cd5ccb9576a54bf1b4a336221b30f74c132344 --- /dev/null +++ b/deploy/models/classifiers/random/printability__stacking_rf_xgb_lr_lr.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:42d86a8fced6c364b61c8a5bff18beeae49c1e11accb2fffc2c52322c6743db6 +size 6043500 diff --git a/deploy/models/classifiers/random/printability__svm_polynomial.pkl b/deploy/models/classifiers/random/printability__svm_polynomial.pkl new file mode 100644 index 0000000000000000000000000000000000000000..008bc4ce57c8ba74a257da5604192ea052b92626 --- /dev/null +++ b/deploy/models/classifiers/random/printability__svm_polynomial.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e1c73dda1047c5a30d2849fff8b56d610e76c04d4f4435bf3bdc294b3c536720 +size 67092 diff --git a/deploy/models/classifiers/random/printability__svm_rbf.pkl b/deploy/models/classifiers/random/printability__svm_rbf.pkl new file mode 100644 index 0000000000000000000000000000000000000000..fd9db930cb6955ae7fb2d6f5d725ac4a74bec9d6 --- /dev/null +++ b/deploy/models/classifiers/random/printability__svm_rbf.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:11f12577a98883c69cb233a01f3682a2e3846e28c2076e34002cd1293d92c9b1 +size 68766 diff --git a/deploy/models/classifiers/random/printability__xgboost.pkl b/deploy/models/classifiers/random/printability__xgboost.pkl new file mode 100644 index 0000000000000000000000000000000000000000..494fc042c99c2e346d16be65f75c425644b6ee95 --- /dev/null +++ b/deploy/models/classifiers/random/printability__xgboost.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:456035d8a5b709323c9025021c4cc405b320ef19c12922d5343d3544f5e31fda +size 651415 diff --git a/deploy/models/deep/random/cell_response__1d_cnn.pt b/deploy/models/deep/random/cell_response__1d_cnn.pt new file mode 100644 index 0000000000000000000000000000000000000000..f6a0e5404e71afaba7c86f02f4c444c6ef114cbc --- /dev/null +++ b/deploy/models/deep/random/cell_response__1d_cnn.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3c8f12a5c725035f14226225d60bda49a4340889d2d5d7701e81f0fece22c60f +size 2255387 diff --git a/deploy/models/deep/random/cell_response__ft_transformer.pt b/deploy/models/deep/random/cell_response__ft_transformer.pt new file mode 100644 index 0000000000000000000000000000000000000000..6ce9a9c3802ebfcfd529455ea7fa73cfac23c863 --- /dev/null +++ b/deploy/models/deep/random/cell_response__ft_transformer.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ae43a34114cd48bd3e3c699cbd5a4f008554a9f0ff4b29c085786a248a617bc +size 10286706 diff --git a/deploy/models/deep/random/cell_response__mlp.pt b/deploy/models/deep/random/cell_response__mlp.pt new file mode 100644 index 0000000000000000000000000000000000000000..6c1e3776115fd9758c926dd0f2c41df33cedaa4d --- /dev/null +++ b/deploy/models/deep/random/cell_response__mlp.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f89dec0eab3df20f4030b36d6d270d27578b3eeb5959638f00cc0f13094fa8ad +size 2459549 diff --git a/deploy/models/deep/random/cell_response__node_lite.pt b/deploy/models/deep/random/cell_response__node_lite.pt new file mode 100644 index 0000000000000000000000000000000000000000..e6773ec283b6f271daf243f3e2afe3f308353476 --- /dev/null +++ b/deploy/models/deep/random/cell_response__node_lite.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:037bb2ce02a8e9a2cdf5289e95dc70715a31581df37ae8de8723cb03b83220c3 +size 745231 diff --git a/deploy/models/deep/random/cell_response__resnet.pt b/deploy/models/deep/random/cell_response__resnet.pt new file mode 100644 index 0000000000000000000000000000000000000000..e60f05ccf6fb40c79d61d4328a7234abcfbfbea8 --- /dev/null +++ b/deploy/models/deep/random/cell_response__resnet.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5035fd746ebd5a80842e547a74ef84b9b6b7839b84f90a387130b0f3f68ec0f9 +size 357543 diff --git a/deploy/models/deep/random/cell_response__tabnet_lite.pt b/deploy/models/deep/random/cell_response__tabnet_lite.pt new file mode 100644 index 0000000000000000000000000000000000000000..9426def103dddad3f05f2e5f3efd92b17c124d00 --- /dev/null +++ b/deploy/models/deep/random/cell_response__tabnet_lite.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f64e7187426e3c8235da619d36604c80f02880b86b973bc701111459818d7a5d +size 3162629 diff --git a/deploy/models/deep/random/printability__1d_cnn.pt b/deploy/models/deep/random/printability__1d_cnn.pt new file mode 100644 index 0000000000000000000000000000000000000000..36c4ed31b3bb8d2de1117e320eb455b2f042c753 --- /dev/null +++ b/deploy/models/deep/random/printability__1d_cnn.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ef3649953646a55eae781c730477fffe126874fc5737ecd3b923ab350f0a652 +size 7139363 diff --git a/deploy/models/deep/random/printability__ft_transformer.pt b/deploy/models/deep/random/printability__ft_transformer.pt new file mode 100644 index 0000000000000000000000000000000000000000..62168559c561c77ec1923b91c789e70e604a1d4e --- /dev/null +++ b/deploy/models/deep/random/printability__ft_transformer.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2efe5b49d2317a1eb319c72c4608da81c25ea0988f30084607ff02e2e5ca9cc8 +size 3817667 diff --git a/deploy/models/deep/random/printability__mlp.pt b/deploy/models/deep/random/printability__mlp.pt new file mode 100644 index 0000000000000000000000000000000000000000..2de77ea09f98aab60dfd666ccd92034675d5bfe0 --- /dev/null +++ b/deploy/models/deep/random/printability__mlp.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:24e5457124f01955414b7b080793e93f9b6efe1b9ec8adddcc8d2bfd80d5b692 +size 2181568 diff --git a/deploy/models/deep/random/printability__node_lite.pt b/deploy/models/deep/random/printability__node_lite.pt new file mode 100644 index 0000000000000000000000000000000000000000..d5155d0544ecbd9674ddf660353652ce9f822689 --- /dev/null +++ b/deploy/models/deep/random/printability__node_lite.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c6cfab0ad5fb11fb0974e8b6c65e36fbfa099c13d0e665e5610fa0f3a4aa4bce +size 576316 diff --git a/deploy/models/deep/random/printability__resnet.pt b/deploy/models/deep/random/printability__resnet.pt new file mode 100644 index 0000000000000000000000000000000000000000..4d7f81db8733f444788ba626185397a7fd1a5b30 --- /dev/null +++ b/deploy/models/deep/random/printability__resnet.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3d4f4f7faa7b5a231e834c8a691601c8e80bb5694adbba815f27be1653ba5b05 +size 753903 diff --git a/deploy/models/deep/random/printability__tabnet_lite.pt b/deploy/models/deep/random/printability__tabnet_lite.pt new file mode 100644 index 0000000000000000000000000000000000000000..0799402fea69743588b0a074ef0ad2e636815e65 --- /dev/null +++ b/deploy/models/deep/random/printability__tabnet_lite.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:65e43d38242eea53dd7709d714d052368c13984a1dfb1bb1c1b7e6c2a55f7c42 +size 1472384 diff --git a/deploy/models/deployment_manifest.json b/deploy/models/deployment_manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..c0af9f0c6865185f38db1224684daebb88c4f466 --- /dev/null +++ b/deploy/models/deployment_manifest.json @@ -0,0 +1,4933 @@ +{ + "created": "2026-09-05", + "selection": "weighted", + "protocols": [ + "random", + "doi" + ], + "fitted_on": "the complete dataset, behind the release preprocessor", + "selected_on": "the tuned configuration for each (task, protocol, model), chosen inside the training partition by 10-fold cross-validation", + "n_artefacts": 154, + "total_bytes": 143804929, + "models": [ + { + "family": "ml", + "task": "cell_response", + "model": "AdaBoost", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__adaboost.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7541, + "params": { + "n_estimators": 231, + "learning_rate": 1.1782013451884574 + }, + "metrics": { + "accuracy": 0.734, + "balanced_accuracy": 0.4229, + "macro_f1": 0.4199, + "weighted_f1": 0.7123, + "quadratic_kappa": 0.7937, + "kappa": 0.5478, + "mcc": 0.5556, + "roc_auc_weighted_ovr": 0.9102 + }, + "bytes": 44341, + "sha256": "8933115563a17d4ba1ce2bcb15daa1564c86e5b697a11a3350c8424fef507f93", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Bagged Trees", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__bagged_trees.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7745, + "params": { + "n_estimators": 133, + "max_samples": 0.9452775017696622, + "max_features": 0.5506389918878342, + "bootstrap": false, + "estimator__max_depth": 18, + "estimator__min_samples_leaf": 4 + }, + "metrics": { + "accuracy": 0.7774, + "balanced_accuracy": 0.5426, + "macro_f1": 0.5554, + "weighted_f1": 0.7729, + "quadratic_kappa": 0.8198, + "kappa": 0.6275, + "mcc": 0.6289, + "roc_auc_weighted_ovr": 0.9531 + }, + "bytes": 1306142, + "sha256": "ec7fd79460aca6c9212d7757da8b19b01f50ef17b35fbb1e4817a935429fd8f3", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Bernoulli Naive Bayes", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__bernoulli_naive_bayes.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7735, + "params": { + "alpha": 0.013244710803878143, + "binarize": 0.004147120866241897, + "fit_prior": true + }, + "metrics": { + "accuracy": 0.7453, + "balanced_accuracy": 0.5193, + "macro_f1": 0.5168, + "weighted_f1": 0.7432, + "quadratic_kappa": 0.7877, + "kappa": 0.5761, + "mcc": 0.5765, + "roc_auc_weighted_ovr": 0.9274 + }, + "bytes": 4218, + "sha256": "282feda9b6e91e96d5dd1ec9cf6ec43d51e8658a38aae0ff94f763ee9e800368", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "CatBoost", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__catboost.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7752, + "params": { + "iterations": 188, + "learning_rate": 0.1629637274346415, + "depth": 5, + "l2_leaf_reg": 4.712189903368168, + "bagging_temperature": 0.38131021691801714, + "random_strength": 1.0191560470062746, + "colsample_bylevel": 0.5323590714017395, + "bootstrap_type": "Bayesian" + }, + "metrics": { + "accuracy": 0.7472, + "balanced_accuracy": 0.4555, + "macro_f1": 0.4552, + "weighted_f1": 0.7289, + "quadratic_kappa": 0.7998, + "kappa": 0.5703, + "mcc": 0.5745, + "roc_auc_weighted_ovr": 0.9513 + }, + "bytes": 110986, + "sha256": "e8acf6b8932614de2bd91250155ade47fe8a9249d4190d24aacb7212f8ec9a9f", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Decision Tree", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__decision_tree.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7755, + "params": { + "criterion": "entropy", + "splitter": "best", + "max_depth": 31, + "min_samples_split": 8, + "min_samples_leaf": 1, + "max_features": null, + "ccp_alpha": 0.0047703143214407025 + }, + "metrics": { + "accuracy": 0.7472, + "balanced_accuracy": 0.466, + "macro_f1": 0.4787, + "weighted_f1": 0.7339, + "quadratic_kappa": 0.77, + "kappa": 0.5697, + "mcc": 0.5738, + "roc_auc_weighted_ovr": 0.9346 + }, + "bytes": 3478, + "sha256": "fec56bf6dad067b23827be89f8a816e5c10871268acce82da80894fbb58a5e25", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Extra Tree", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__extra_tree.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7508, + "params": { + "criterion": "log_loss", + "max_depth": 22, + "min_samples_split": 14, + "min_samples_leaf": 13, + "max_features": null, + "ccp_alpha": 0.004314149197954582 + }, + "metrics": { + "accuracy": 0.7396, + "balanced_accuracy": 0.4076, + "macro_f1": 0.3846, + "weighted_f1": 0.712, + "quadratic_kappa": 0.82, + "kappa": 0.5575, + "mcc": 0.5642, + "roc_auc_weighted_ovr": 0.9332 + }, + "bytes": 3469, + "sha256": "0128ac01ef824a6b8df6f745106d695b6a593a292666ae517d3d68924d514153", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Extra Trees", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__extra_trees.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7756, + "params": { + "n_estimators": 372, + "criterion": "entropy", + "max_depth": 53, + "min_samples_split": 2, + "min_samples_leaf": 3, + "max_features": null, + "bootstrap": true, + "ccp_alpha": 0.0034748638522106978 + }, + "metrics": { + "accuracy": 0.7453, + "balanced_accuracy": 0.4813, + "macro_f1": 0.4975, + "weighted_f1": 0.7335, + "quadratic_kappa": 0.7441, + "kappa": 0.5649, + "mcc": 0.5668, + "roc_auc_weighted_ovr": 0.9516 + }, + "bytes": 1600493, + "sha256": "2a1ebaf96b9d13fcc91b5a3c1ca09633eab16f7b33162624917e435cde2f2efd", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Gaussian Naive Bayes", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__gaussian_naive_bayes.pkl", + "source_fold": "random_holdout", + "inner_score": 0.3015, + "params": { + "var_smoothing": 8.75017262089054e-06 + }, + "metrics": { + "accuracy": 0.2698, + "balanced_accuracy": 0.401, + "macro_f1": 0.26, + "weighted_f1": 0.3115, + "quadratic_kappa": 0.3386, + "kappa": 0.1436, + "mcc": 0.1964, + "roc_auc_weighted_ovr": 0.8078 + }, + "bytes": 7337, + "sha256": "d00e8cc7debd29d176894f559c3b043bd0ef03c6b1d0ecc6c86fa2149f6855d0", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Gradient Boosting", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__gradient_boosting.pkl", + "source_fold": "random_holdout", + "inner_score": 0.69, + "params": { + "learning_rate": 0.04187594718900627, + "n_estimators": 164, + "subsample": 0.864803089169032, + "criterion": "squared_error", + "min_samples_split": 10, + "min_samples_leaf": 3, + "max_depth": 15, + "max_features": null, + "ccp_alpha": 0.02468977981821954 + }, + "metrics": { + "accuracy": 0.7358, + "balanced_accuracy": 0.385, + "macro_f1": 0.3083, + "weighted_f1": 0.6671, + "quadratic_kappa": 0.7765, + "kappa": 0.5473, + "mcc": 0.5899, + "roc_auc_weighted_ovr": 0.8972 + }, + "bytes": 28636, + "sha256": "0a4332f43c8af0932c05662b97fde1f0d67525773457cd72ddcc670273462767", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Hist Gradient Boosting", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__hist_gradient_boosting.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7831, + "params": { + "learning_rate": 0.017232330428016588, + "max_iter": 299, + "max_leaf_nodes": 11, + "max_depth": 29, + "min_samples_leaf": 8, + "l2_regularization": 0.6713320214175605, + "max_bins": 103, + "early_stopping": false, + "n_iter_no_change": 10 + }, + "metrics": { + "accuracy": 0.7698, + "balanced_accuracy": 0.5317, + "macro_f1": 0.5407, + "weighted_f1": 0.7674, + "quadratic_kappa": 0.8242, + "kappa": 0.6163, + "mcc": 0.6172, + "roc_auc_weighted_ovr": 0.9569 + }, + "bytes": 645934, + "sha256": "2bf42d2ad81b2b130b6d295e9d97480d9ada674e89d701575c2bf7be6d379ecb", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "LightGBM", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__lightgbm.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7749, + "params": { + "boosting_type": "gbdt", + "num_leaves": 234, + "max_depth": 36, + "learning_rate": 0.03878772651508437, + "n_estimators": 380, + "min_child_weight": 3.6500492398306026, + "min_child_samples": 29, + "subsample": 0.5671329236007874, + "subsample_freq": 4, + "colsample_bytree": 0.9308753250964105, + "reg_alpha": 4.430439770434747, + "reg_lambda": 3.2039662929835404 + }, + "metrics": { + "accuracy": 0.7434, + "balanced_accuracy": 0.4387, + "macro_f1": 0.4313, + "weighted_f1": 0.7267, + "quadratic_kappa": 0.8194, + "kappa": 0.5668, + "mcc": 0.5695, + "roc_auc_weighted_ovr": 0.9466 + }, + "bytes": 552411, + "sha256": "2c198a22bf0a01258f0e9196472c74797ae0529ab244b1c3e0fe0d6c40e1d23a", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Linear Discriminant", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__linear_discriminant.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7502, + "params": { + "shrinkage": 0.5, + "n_components": 3, + "tol": 0.000684792009557478 + }, + "metrics": { + "accuracy": 0.7113, + "balanced_accuracy": 0.4198, + "macro_f1": 0.4345, + "weighted_f1": 0.6865, + "quadratic_kappa": 0.6429, + "kappa": 0.4887, + "mcc": 0.494, + "roc_auc_weighted_ovr": 0.9254 + }, + "bytes": 145025, + "sha256": "14d6b11c6b801eef3e7f9fa0c6ad28c0a9366fad15cce208bd00725d313adcb2", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Linear SVM", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__linear_svm.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7695, + "params": { + "loss": "hinge", + "tol": 0.0017730095882880273, + "C": 0.28331414232062857, + "fit_intercept": true, + "intercept_scaling": 0.15066341859805293, + "max_iter": 3874, + "class_weight": "balanced" + }, + "metrics": { + "accuracy": 0.7208, + "balanced_accuracy": 0.4457, + "macro_f1": 0.4531, + "weighted_f1": 0.7032, + "quadratic_kappa": 0.7005, + "kappa": 0.5166, + "mcc": 0.52 + }, + "bytes": 5535, + "sha256": "6f41c3e869a8250cbbf74ea42271cd3a07684d46794b48a1ff042f29cf8cd9d9", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Logistic Regression", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__logistic_regression.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7566, + "params": { + "penalty": "l2", + "tol": 0.0010434843362977706, + "C": 2.9482840286713903, + "fit_intercept": true, + "solver": "lbfgs", + "max_iter": 1695 + }, + "metrics": { + "accuracy": 0.7151, + "balanced_accuracy": 0.4051, + "macro_f1": 0.405, + "weighted_f1": 0.6845, + "quadratic_kappa": 0.6747, + "kappa": 0.4962, + "mcc": 0.5017, + "roc_auc_weighted_ovr": 0.9248 + }, + "bytes": 6535, + "sha256": "fe67a6655375a6e0bfcc706115cc992ed566ba87e57a5696f22843b1b681c1a9", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Logistic Regression (balanced)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__logistic_regression_balanced.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7535, + "params": { + "penalty": null, + "tol": 0.006999971755699955, + "C": 0.22829596483959813, + "fit_intercept": false, + "solver": "lbfgs", + "max_iter": 1490 + }, + "metrics": { + "accuracy": 0.7245, + "balanced_accuracy": 0.4798, + "macro_f1": 0.4665, + "weighted_f1": 0.7287, + "quadratic_kappa": 0.809, + "kappa": 0.5462, + "mcc": 0.5499, + "roc_auc_weighted_ovr": 0.9354 + }, + "bytes": 6514, + "sha256": "fc90e1018ef9d648de7adb5da1b60e267f4b1e357f538b8fdaa5c6b246c0c97d", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "MLP (256-128)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__mlp_256_128.pkl", + "source_fold": "random_holdout", + "inner_score": 0.768, + "params": { + "hidden_layer_sizes": "128_64", + "activation": "logistic", + "solver": "adam", + "alpha": 0.012656977485848033, + "learning_rate": "invscaling", + "learning_rate_init": 0.00019998095347852762, + "max_iter": 964, + "tol": 2.4837224679225704e-05, + "early_stopping": false + }, + "metrics": { + "accuracy": 0.7302, + "balanced_accuracy": 0.4305, + "macro_f1": 0.4271, + "weighted_f1": 0.703, + "quadratic_kappa": 0.7048, + "kappa": 0.5288, + "mcc": 0.5332, + "roc_auc_weighted_ovr": 0.9242 + }, + "bytes": 877150, + "sha256": "2f4bf03e58aea3bb7c171b22d4bf03077652236328dcbf30317765bd28c95a35", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Nearest Centroid", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__nearest_centroid.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7277, + "params": { + "metric": "euclidean", + "shrink_threshold": 0.9941279891975694 + }, + "metrics": { + "accuracy": 0.6962, + "balanced_accuracy": 0.4824, + "macro_f1": 0.4397, + "weighted_f1": 0.679, + "quadratic_kappa": 0.6847, + "kappa": 0.4876, + "mcc": 0.4986, + "roc_auc_weighted_ovr": 0.9253 + }, + "bytes": 7171, + "sha256": "579188d8f4730741cc6d4fcec2d042de26c5f1c4e9810ab23851e2152ec467e3", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Passive Aggressive", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__passive_aggressive.pkl", + "source_fold": "random_holdout", + "inner_score": 0.767, + "params": { + "C": 0.013425018130283161, + "fit_intercept": false, + "max_iter": 2475, + "tol": 3.355037266594036e-05, + "early_stopping": false, + "validation_fraction": 0.27798997875789383, + "n_iter_no_change": 5, + "shuffle": true, + "loss": "hinge", + "class_weight": "balanced", + "average": true + }, + "metrics": { + "accuracy": 0.7151, + "balanced_accuracy": 0.4313, + "macro_f1": 0.4351, + "weighted_f1": 0.696, + "quadratic_kappa": 0.6917, + "kappa": 0.5053, + "mcc": 0.5091 + }, + "bytes": 12296, + "sha256": "6d8c577687544c0044204267fce8e94a8fcd695be0e7d8e4660e029bf2b409f7", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Perceptron", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__perceptron.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7248, + "params": { + "penalty": "elasticnet", + "alpha": 5.1407142801356526e-05, + "l1_ratio": 0.6344887536320425, + "fit_intercept": true, + "max_iter": 1082, + "tol": 0.0005236045454094717, + "shuffle": false, + "eta0": 0.4787761815206933, + "early_stopping": false, + "validation_fraction": 0.29793290974018816, + "n_iter_no_change": 9, + "class_weight": null + }, + "metrics": { + "accuracy": 0.6906, + "balanced_accuracy": 0.3888, + "macro_f1": 0.3847, + "weighted_f1": 0.6549, + "quadratic_kappa": 0.6081, + "kappa": 0.4438, + "mcc": 0.453 + }, + "bytes": 4111, + "sha256": "12bd23ecb4a5e58d8a3ba47a549bc38b7e9d027a1dde92c474a5e673b8cd3d34", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Quadratic Discriminant", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__quadratic_discriminant.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7694, + "params": { + "pca__n_components": 13, + "reg_param": 0.01803192183395734, + "store_covariance": true, + "tol": 0.0061904445700914235 + }, + "metrics": { + "accuracy": 0.7509, + "balanced_accuracy": 0.5069, + "macro_f1": 0.5131, + "weighted_f1": 0.7494, + "quadratic_kappa": 0.816, + "kappa": 0.5854, + "mcc": 0.5857, + "roc_auc_weighted_ovr": 0.9348 + }, + "bytes": 29521, + "sha256": "006ecba15ba6131d1dff15163c6168615363be3716e8eb3fc96a4b63b0f44fc7", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Random Forest", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__random_forest.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7382, + "params": { + "n_estimators": 300, + "criterion": "entropy", + "max_depth": 23, + "min_samples_split": 6, + "min_samples_leaf": 8, + "min_weight_fraction_leaf": 0.13609734250171107, + "max_features": null, + "bootstrap": false, + "ccp_alpha": 0.019326628857371634, + "max_samples": null + }, + "metrics": { + "accuracy": 0.7321, + "balanced_accuracy": 0.3988, + "macro_f1": 0.3641, + "weighted_f1": 0.7002, + "quadratic_kappa": 0.7967, + "kappa": 0.5457, + "mcc": 0.5558, + "roc_auc_weighted_ovr": 0.9055 + }, + "bytes": 4994, + "sha256": "1b5b4c7ddd69ecf0575e2cad3dc3b94992ded2934f1fbc74498182984e396165", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Random Forest (balanced)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__random_forest_balanced.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7736, + "params": { + "n_estimators": 500, + "criterion": "log_loss", + "max_depth": 10, + "min_samples_split": 3, + "min_samples_leaf": 4, + "min_weight_fraction_leaf": 0.003049996444172705, + "max_features": "sqrt", + "bootstrap": false, + "ccp_alpha": 0.00027927046656537474, + "max_samples": 0.8 + }, + "metrics": { + "accuracy": 0.7642, + "balanced_accuracy": 0.6028, + "macro_f1": 0.5605, + "weighted_f1": 0.7663, + "quadratic_kappa": 0.8182, + "kappa": 0.6154, + "mcc": 0.6194, + "roc_auc_weighted_ovr": 0.9464 + }, + "bytes": 3902841, + "sha256": "7be9e347e9d4921977931954bf9d2326544452d96d7bcd2e9ddbfae163a95b77", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Ridge Classifier", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__ridge_classifier.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7086, + "params": { + "alpha": 0.01699839043018258, + "fit_intercept": true, + "max_iter": 1413, + "tol": 1.1595408255619176e-05, + "solver": "saga" + }, + "metrics": { + "accuracy": 0.6906, + "balanced_accuracy": 0.3754, + "macro_f1": 0.3745, + "weighted_f1": 0.6479, + "quadratic_kappa": 0.6137, + "kappa": 0.4362, + "mcc": 0.4471 + }, + "bytes": 6651, + "sha256": "887429d6d2f49ef7c671d9be75c29bb94bc1c7e3abbe202dba13596533c96af9", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "SGD (hinge)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__sgd_hinge.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7583, + "params": { + "penalty": "l1", + "alpha": 7.034984795850037e-05, + "l1_ratio": 0.29856284659902343, + "fit_intercept": true, + "max_iter": 614, + "tol": 4.77764214879311e-05, + "learning_rate": "adaptive", + "eta0": 0.08855603039428027, + "early_stopping": false, + "validation_fraction": 0.1619562341957141, + "n_iter_no_change": 2 + }, + "metrics": { + "accuracy": 0.717, + "balanced_accuracy": 0.4069, + "macro_f1": 0.4061, + "weighted_f1": 0.6864, + "quadratic_kappa": 0.6874, + "kappa": 0.5004, + "mcc": 0.506 + }, + "bytes": 4886, + "sha256": "0070b38894cb7ef91f72dae63d7efe23ce835ee369e481d3b366062c886d3baa", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "SVM (RBF)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__svm_rbf.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7637, + "params": { + "C": 1.703442725584415, + "gamma": "scale", + "tol": 0.007922017270530962, + "shrinking": true, + "class_weight": null, + "decision_function_shape": "ovo" + }, + "metrics": { + "accuracy": 0.7057, + "balanced_accuracy": 0.3962, + "macro_f1": 0.398, + "weighted_f1": 0.6815, + "quadratic_kappa": 0.6976, + "kappa": 0.4878, + "mcc": 0.4913, + "roc_auc_weighted_ovr": 0.9219 + }, + "bytes": 66004, + "sha256": "ec357aa39dbb6631b933c1e98e4d4b2df6135cc5237a40b2d24dfb6c7d128e45", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "SVM (polynomial)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__svm_polynomial.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7677, + "params": { + "C": 0.8217470561673981, + "gamma": "scale", + "tol": 0.006748675390170305, + "shrinking": true, + "class_weight": null, + "decision_function_shape": "ovo", + "degree": 2, + "coef0": 0.1543457973837144 + }, + "metrics": { + "accuracy": 0.7151, + "balanced_accuracy": 0.396, + "macro_f1": 0.3932, + "weighted_f1": 0.6846, + "quadratic_kappa": 0.7016, + "kappa": 0.5004, + "mcc": 0.506, + "roc_auc_weighted_ovr": 0.9228 + }, + "bytes": 60088, + "sha256": "bd644c182976274d6ec346169b332e3f18d29912c2152e91f3b6ae17c3155a34", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Soft Voting (RF+XGB+LR)", + "protocol": "random", + "selection": "composed", + "file": "classifiers/random/cell_response__soft_voting_rf_xgb_lr.pkl", + "source_fold": "random_holdout", + "inner_score": null, + "params": { + "composed_from": [ + "Logistic Regression", + "Random Forest", + "XGBoost" + ] + }, + "metrics": { + "accuracy": 0.7585, + "balanced_accuracy": 0.4919, + "macro_f1": 0.5044, + "weighted_f1": 0.7444, + "quadratic_kappa": 0.7885, + "kappa": 0.5881, + "mcc": 0.5914, + "roc_auc_weighted_ovr": 0.95 + }, + "bytes": 5233168, + "sha256": "19f3ea953fb061519b7d0fa42416c8edacffc93f762e87cab7a90f5692643340", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Stacking (RF+XGB+LR -> LR)", + "protocol": "random", + "selection": "composed", + "file": "classifiers/random/cell_response__stacking_rf_xgb_lr_lr.pkl", + "source_fold": "random_holdout", + "inner_score": null, + "params": { + "composed_from": [ + "Logistic Regression", + "Random Forest", + "XGBoost" + ] + }, + "metrics": { + "accuracy": 0.7849, + "balanced_accuracy": 0.5358, + "macro_f1": 0.5542, + "weighted_f1": 0.7735, + "quadratic_kappa": 0.8186, + "kappa": 0.6361, + "mcc": 0.6413, + "roc_auc_weighted_ovr": 0.9486 + }, + "bytes": 5233839, + "sha256": "6ed798ca8edf3a2f54aae8fde0cc0aaa3eba4457897b97ffa19be49c85c035ba", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "XGBoost", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__xgboost.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7738, + "params": { + "n_estimators": 297, + "max_depth": 4, + "learning_rate": 0.07498039486034523, + "gamma": 1.4181682661295807, + "min_child_weight": 1.8432503737904644, + "subsample": 0.524250824250758, + "colsample_bytree": 0.5170505017122972, + "reg_alpha": 0.025940403490796626, + "reg_lambda": 2.6902781605693025 + }, + "metrics": { + "accuracy": 0.7396, + "balanced_accuracy": 0.4447, + "macro_f1": 0.4374, + "weighted_f1": 0.7243, + "quadratic_kappa": 0.8142, + "kappa": 0.5606, + "mcc": 0.5629, + "roc_auc_weighted_ovr": 0.9482 + }, + "bytes": 189666, + "sha256": "e9408add167a4c715a875fd88daf4457c46f058ba91c8fa6f9ca57ce1b7d9bb8", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "k-NN (distance weighted)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__k_nn_distance_weighted.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7492, + "params": { + "n_neighbors": 2, + "algorithm": "kd_tree", + "leaf_size": 30, + "p": 2, + "metric": "minkowski" + }, + "metrics": { + "accuracy": 0.7415, + "balanced_accuracy": 0.5231, + "macro_f1": 0.5237, + "weighted_f1": 0.7338, + "quadratic_kappa": 0.7408, + "kappa": 0.5629, + "mcc": 0.5648, + "roc_auc_weighted_ovr": 0.8354 + }, + "bytes": 127106, + "sha256": "534a360ee86a358052a823d1107c6c002e4e9ef58a3e84f1c2c860f025c2222d", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "k-Nearest Neighbours", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/cell_response__k_nearest_neighbours.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7461, + "params": { + "n_neighbors": 1, + "algorithm": "auto", + "leaf_size": 52, + "p": 2, + "metric": "euclidean" + }, + "metrics": { + "accuracy": 0.7302, + "balanced_accuracy": 0.4945, + "macro_f1": 0.4946, + "weighted_f1": 0.7215, + "quadratic_kappa": 0.742, + "kappa": 0.5434, + "mcc": 0.5449, + "roc_auc_weighted_ovr": 0.8195 + }, + "bytes": 53470, + "sha256": "780fbe925315a38f89bbf907dac7e4b1f587266533df7c0115e0fc04276b40a0", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "AdaBoost", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__adaboost.pkl", + "source_fold": "random_holdout", + "inner_score": 0.6138, + "params": { + "n_estimators": 271, + "learning_rate": 1.7549303968840675 + }, + "metrics": { + "accuracy": 0.6377, + "balanced_accuracy": 0.5121, + "macro_f1": 0.5137, + "weighted_f1": 0.6285, + "quadratic_kappa": 0.5031, + "kappa": 0.4322, + "mcc": 0.4336, + "roc_auc_weighted_ovr": 0.7692 + }, + "bytes": 48212, + "sha256": "61fa142eee65a9a3dab6e40de5ec7715340c56563254ca52623e45f0a3681625", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Bagged Trees", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__bagged_trees.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7677, + "params": { + "n_estimators": 193, + "max_samples": 0.7822899911930258, + "max_features": 0.8617320922989526, + "bootstrap": false, + "estimator__max_depth": 29, + "estimator__min_samples_leaf": 1 + }, + "metrics": { + "accuracy": 0.7943, + "balanced_accuracy": 0.7415, + "macro_f1": 0.7627, + "weighted_f1": 0.7922, + "quadratic_kappa": 0.8012, + "kappa": 0.6763, + "mcc": 0.6775, + "roc_auc_weighted_ovr": 0.915 + }, + "bytes": 3775738, + "sha256": "98d12f997b7e7bcb6789ddbf32dacddcc53c859e265778f18682b7b693397e51", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Bernoulli Naive Bayes", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__bernoulli_naive_bayes.pkl", + "source_fold": "random_holdout", + "inner_score": 0.6176, + "params": { + "alpha": 0.09984303783310336, + "binarize": 0.07979497095883875, + "fit_prior": true + }, + "metrics": { + "accuracy": 0.6415, + "balanced_accuracy": 0.5659, + "macro_f1": 0.5362, + "weighted_f1": 0.6425, + "quadratic_kappa": 0.5704, + "kappa": 0.4592, + "mcc": 0.4628, + "roc_auc_weighted_ovr": 0.8188 + }, + "bytes": 3497, + "sha256": "98d82bc7b87ea7a6e51932768636f4f9cb2e62d7f796d1f1f2aa420230ae0071", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "CatBoost", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__catboost.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7679, + "params": { + "iterations": 483, + "learning_rate": 0.2820482852343135, + "depth": 10, + "l2_leaf_reg": 4.368204366176838, + "bagging_temperature": 0.40835632276528555, + "random_strength": 2.8251241143568477, + "colsample_bylevel": 0.5491147709329218, + "bootstrap_type": "Bayesian" + }, + "metrics": { + "accuracy": 0.7887, + "balanced_accuracy": 0.7369, + "macro_f1": 0.7539, + "weighted_f1": 0.7864, + "quadratic_kappa": 0.7958, + "kappa": 0.6681, + "mcc": 0.6691, + "roc_auc_weighted_ovr": 0.9183 + }, + "bytes": 787008, + "sha256": "45c920875ad42ed56cade9561ab095e950e209819d68f2e061ff68534dd73fd3", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Decision Tree", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__decision_tree.pkl", + "source_fold": "random_holdout", + "inner_score": 0.6996, + "params": { + "criterion": "entropy", + "splitter": "best", + "max_depth": 16, + "min_samples_split": 15, + "min_samples_leaf": 7, + "max_features": null, + "ccp_alpha": 0.0011685585033690445 + }, + "metrics": { + "accuracy": 0.7132, + "balanced_accuracy": 0.6553, + "macro_f1": 0.6534, + "weighted_f1": 0.7116, + "quadratic_kappa": 0.6497, + "kappa": 0.5554, + "mcc": 0.5557, + "roc_auc_weighted_ovr": 0.8866 + }, + "bytes": 10029, + "sha256": "768ec1872e1fc323154af785e6f71d554389de56fb43aa2c0624e6a70852a6b5", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Extra Tree", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__extra_tree.pkl", + "source_fold": "random_holdout", + "inner_score": 0.6466, + "params": { + "criterion": "log_loss", + "max_depth": 33, + "min_samples_split": 7, + "min_samples_leaf": 12, + "max_features": null, + "ccp_alpha": 0.0003790042628302472 + }, + "metrics": { + "accuracy": 0.6585, + "balanced_accuracy": 0.5668, + "macro_f1": 0.5563, + "weighted_f1": 0.6487, + "quadratic_kappa": 0.5113, + "kappa": 0.4652, + "mcc": 0.4683, + "roc_auc_weighted_ovr": 0.8628 + }, + "bytes": 9601, + "sha256": "591f152e86031f04d9d92a5cd88164dcb636509e63d6726010c6814b512bfa75", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Extra Trees", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__extra_trees.pkl", + "source_fold": "random_holdout", + "inner_score": 0.735, + "params": { + "n_estimators": 61, + "criterion": "entropy", + "max_depth": 60, + "min_samples_split": 17, + "min_samples_leaf": 1, + "max_features": null, + "bootstrap": false, + "ccp_alpha": 0.0020050460186222235 + }, + "metrics": { + "accuracy": 0.7604, + "balanced_accuracy": 0.701, + "macro_f1": 0.7131, + "weighted_f1": 0.7602, + "quadratic_kappa": 0.6732, + "kappa": 0.6269, + "mcc": 0.6282, + "roc_auc_weighted_ovr": 0.9098 + }, + "bytes": 487961, + "sha256": "e152e1e7423d5e3306b7abe26ee13ed67596433b1e910f818ed6840a284fffa7", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Gaussian Naive Bayes", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__gaussian_naive_bayes.pkl", + "source_fold": "random_holdout", + "inner_score": 0.3059, + "params": { + "var_smoothing": 9.936989689136058e-06 + }, + "metrics": { + "accuracy": 0.2792, + "balanced_accuracy": 0.4067, + "macro_f1": 0.2763, + "weighted_f1": 0.3055, + "quadratic_kappa": 0.1707, + "kappa": 0.1353, + "mcc": 0.1813, + "roc_auc_weighted_ovr": 0.7057 + }, + "bytes": 6142, + "sha256": "d806b6407b0cc8ab12d7fa836dee8811e349c8e740b7983016111b02d2239223", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Gradient Boosting", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__gradient_boosting.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7396, + "params": { + "learning_rate": 0.029956893008933304, + "n_estimators": 378, + "subsample": 0.753883338314261, + "criterion": "squared_error", + "min_samples_split": 2, + "min_samples_leaf": 6, + "max_depth": 12, + "max_features": "sqrt", + "ccp_alpha": 0.0001345429344379672 + }, + "metrics": { + "accuracy": 0.766, + "balanced_accuracy": 0.6972, + "macro_f1": 0.7126, + "weighted_f1": 0.7629, + "quadratic_kappa": 0.6859, + "kappa": 0.6314, + "mcc": 0.6331, + "roc_auc_weighted_ovr": 0.9137 + }, + "bytes": 955239, + "sha256": "47a1931819018e107c00db7279b2e46722b92606c7d7fc1bf95f5f2728e9e7fa", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Hist Gradient Boosting", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__hist_gradient_boosting.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7669, + "params": { + "learning_rate": 0.3649609080693064, + "max_iter": 124, + "max_leaf_nodes": 67, + "max_depth": 24, + "min_samples_leaf": 6, + "l2_regularization": 0.5274651728863381, + "max_bins": 214, + "early_stopping": false, + "n_iter_no_change": 5 + }, + "metrics": { + "accuracy": 0.7868, + "balanced_accuracy": 0.7487, + "macro_f1": 0.7576, + "weighted_f1": 0.7843, + "quadratic_kappa": 0.8126, + "kappa": 0.6664, + "mcc": 0.6672, + "roc_auc_weighted_ovr": 0.9169 + }, + "bytes": 1555799, + "sha256": "8503f4f620cc539db9e017106b8df556958c3e3191c60da4568d428ac0e2f64f", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "LightGBM", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__lightgbm.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7554, + "params": { + "boosting_type": "gbdt", + "num_leaves": 98, + "max_depth": 31, + "learning_rate": 0.13822583920308426, + "n_estimators": 377, + "min_child_weight": 1.6897274615450322, + "min_child_samples": 30, + "subsample": 0.6668421259112413, + "subsample_freq": 5, + "colsample_bytree": 0.6742471046707172, + "reg_alpha": 0.19261465401965583, + "reg_lambda": 1.2666428268293457 + }, + "metrics": { + "accuracy": 0.7642, + "balanced_accuracy": 0.7082, + "macro_f1": 0.7119, + "weighted_f1": 0.7619, + "quadratic_kappa": 0.7879, + "kappa": 0.6328, + "mcc": 0.6332, + "roc_auc_weighted_ovr": 0.9139 + }, + "bytes": 1783453, + "sha256": "59b18f86de402feafc810534cd093c2fe77af87772a1a20c37d050fbc643a1cc", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Linear Discriminant", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__linear_discriminant.pkl", + "source_fold": "random_holdout", + "inner_score": 0.6221, + "params": { + "shrinkage": 0.1, + "n_components": 10, + "tol": 0.00788671412999049 + }, + "metrics": { + "accuracy": 0.6755, + "balanced_accuracy": 0.5515, + "macro_f1": 0.5591, + "weighted_f1": 0.6605, + "quadratic_kappa": 0.5274, + "kappa": 0.4807, + "mcc": 0.4852, + "roc_auc_weighted_ovr": 0.8344 + }, + "bytes": 142568, + "sha256": "7fc2a8a5bf25dc66cabbcc83dc091e8e36d9f7457ca09595f0ca7abc8086342f", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Linear SVM", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__linear_svm.pkl", + "source_fold": "random_holdout", + "inner_score": 0.6268, + "params": { + "loss": "squared_hinge", + "tol": 0.0004337854257719927, + "C": 5.266137427449793, + "fit_intercept": false, + "intercept_scaling": 0.16317458581616753, + "max_iter": 1287, + "class_weight": "balanced" + }, + "metrics": { + "accuracy": 0.634, + "balanced_accuracy": 0.6027, + "macro_f1": 0.5509, + "weighted_f1": 0.6458, + "quadratic_kappa": 0.5861, + "kappa": 0.463, + "mcc": 0.4694 + }, + "bytes": 5296, + "sha256": "5003a237f95531955627e47a8b231ebc28e5ac24a3e4a9ba7d6f36b63d2ee226", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Logistic Regression", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__logistic_regression.pkl", + "source_fold": "random_holdout", + "inner_score": 0.6363, + "params": { + "penalty": null, + "tol": 0.00010976829592516031, + "C": 91.0497240599568, + "fit_intercept": false, + "solver": "lbfgs", + "max_iter": 735 + }, + "metrics": { + "accuracy": 0.6717, + "balanced_accuracy": 0.5905, + "macro_f1": 0.5744, + "weighted_f1": 0.6652, + "quadratic_kappa": 0.5528, + "kappa": 0.4876, + "mcc": 0.4921, + "roc_auc_weighted_ovr": 0.8286 + }, + "bytes": 5402, + "sha256": "e997dcc201a4019f483df41d3c34b500e5ddc28a3c07fa6127a484128f49ce14", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Logistic Regression (balanced)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__logistic_regression_balanced.pkl", + "source_fold": "random_holdout", + "inner_score": 0.6029, + "params": { + "penalty": null, + "tol": 0.0027612366691185343, + "C": 0.01894372701657312, + "fit_intercept": true, + "solver": "saga", + "max_iter": 1069 + }, + "metrics": { + "accuracy": 0.6189, + "balanced_accuracy": 0.6111, + "macro_f1": 0.552, + "weighted_f1": 0.6365, + "quadratic_kappa": 0.5808, + "kappa": 0.4519, + "mcc": 0.4602, + "roc_auc_weighted_ovr": 0.8164 + }, + "bytes": 5487, + "sha256": "6f60a3bf8190a0775b7e2cea7a65e037e9ead9001675af5ff2ed8c10e672749f", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "MLP (256-128)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__mlp_256_128.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7367, + "params": { + "hidden_layer_sizes": "256_128_64", + "activation": "tanh", + "solver": "adam", + "alpha": 9.245123544012322e-05, + "learning_rate": "constant", + "learning_rate_init": 0.0026461810673864042, + "max_iter": 816, + "tol": 1.0723297771337903e-06, + "early_stopping": false + }, + "metrics": { + "accuracy": 0.7415, + "balanced_accuracy": 0.6977, + "macro_f1": 0.7058, + "weighted_f1": 0.7417, + "quadratic_kappa": 0.7449, + "kappa": 0.5998, + "mcc": 0.6, + "roc_auc_weighted_ovr": 0.8765 + }, + "bytes": 2479623, + "sha256": "a8aa16d753eb9aa0e2cb19177aec37ed56c01e9e714fd41977a4a9ddc303c892", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Nearest Centroid", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__nearest_centroid.pkl", + "source_fold": "random_holdout", + "inner_score": 0.4894, + "params": { + "metric": "euclidean", + "shrink_threshold": 0.005770218291259177 + }, + "metrics": { + "accuracy": 0.4642, + "balanced_accuracy": 0.4687, + "macro_f1": 0.4043, + "weighted_f1": 0.4995, + "quadratic_kappa": 0.3778, + "kappa": 0.269, + "mcc": 0.2855, + "roc_auc_weighted_ovr": 0.8096 + }, + "bytes": 10675, + "sha256": "3168f56946288cec73141c20b257408f14c852bd82c626b3387d35427b76297e", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Passive Aggressive", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__passive_aggressive.pkl", + "source_fold": "random_holdout", + "inner_score": 0.6302, + "params": { + "C": 25.273624580658733, + "fit_intercept": false, + "max_iter": 2073, + "tol": 0.08200442203212223, + "early_stopping": true, + "validation_fraction": 0.161658483756386, + "n_iter_no_change": 6, + "shuffle": true, + "loss": "squared_hinge", + "class_weight": "balanced", + "average": true + }, + "metrics": { + "accuracy": 0.6453, + "balanced_accuracy": 0.5998, + "macro_f1": 0.5594, + "weighted_f1": 0.6503, + "quadratic_kappa": 0.5455, + "kappa": 0.469, + "mcc": 0.4737 + }, + "bytes": 10140, + "sha256": "6be8bfb9c6800ce38759f3d005afa519170f8cb4aa0f32239f1e9c42e5b29bc8", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Perceptron", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__perceptron.pkl", + "source_fold": "random_holdout", + "inner_score": 0.5555, + "params": { + "penalty": "l1", + "alpha": 4.311623825732279e-05, + "l1_ratio": 0.959436622288464, + "fit_intercept": true, + "max_iter": 1691, + "tol": 6.119417887934741e-05, + "shuffle": true, + "eta0": 0.19986619824419047, + "early_stopping": false, + "validation_fraction": 0.27280241755001244, + "n_iter_no_change": 6, + "class_weight": null + }, + "metrics": { + "accuracy": 0.5358, + "balanced_accuracy": 0.3435, + "macro_f1": 0.3197, + "weighted_f1": 0.4388, + "quadratic_kappa": 0.2399, + "kappa": 0.1433, + "mcc": 0.1985 + }, + "bytes": 3788, + "sha256": "495b685690763c2aab0212fa584ce4e145709cf8a44479b760a320f1e13a8f25", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Quadratic Discriminant", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__quadratic_discriminant.pkl", + "source_fold": "random_holdout", + "inner_score": 0.5975, + "params": { + "pca__n_components": 50, + "reg_param": 0.3046137691733707, + "store_covariance": false, + "tol": 4.3743644399390745e-05 + }, + "metrics": { + "accuracy": 0.6189, + "balanced_accuracy": 0.5362, + "macro_f1": 0.513, + "weighted_f1": 0.6204, + "quadratic_kappa": 0.5446, + "kappa": 0.4306, + "mcc": 0.4389, + "roc_auc_weighted_ovr": 0.8164 + }, + "bytes": 140334, + "sha256": "7424e80ca7b551781302e507fe091b26669c442d5fb984ae60b456cd23d50f6b", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Random Forest", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__random_forest.pkl", + "source_fold": "random_holdout", + "inner_score": 0.5634, + "params": { + "n_estimators": 300, + "criterion": "log_loss", + "max_depth": 8, + "min_samples_split": 5, + "min_samples_leaf": 10, + "min_weight_fraction_leaf": 0.06238413725820846, + "max_features": null, + "bootstrap": true, + "ccp_alpha": 0.007654621119809567, + "max_samples": 0.8 + }, + "metrics": { + "accuracy": 0.6321, + "balanced_accuracy": 0.4528, + "macro_f1": 0.4351, + "weighted_f1": 0.5928, + "quadratic_kappa": 0.3945, + "kappa": 0.4081, + "mcc": 0.4229, + "roc_auc_weighted_ovr": 0.8216 + }, + "bytes": 262948, + "sha256": "733cbf8260af6d48d37e4957131e63a12c3f97a0f8af09d7dd4671c27084bbd3", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Random Forest (balanced)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__random_forest_balanced.pkl", + "source_fold": "random_holdout", + "inner_score": 0.6913, + "params": { + "n_estimators": 450, + "criterion": "entropy", + "max_depth": 16, + "min_samples_split": 5, + "min_samples_leaf": 4, + "min_weight_fraction_leaf": 0.0037799050709445034, + "max_features": null, + "bootstrap": true, + "ccp_alpha": 0.006897076648491601, + "max_samples": 0.8 + }, + "metrics": { + "accuracy": 0.7208, + "balanced_accuracy": 0.7199, + "macro_f1": 0.675, + "weighted_f1": 0.7293, + "quadratic_kappa": 0.6495, + "kappa": 0.5893, + "mcc": 0.5942, + "roc_auc_weighted_ovr": 0.8944 + }, + "bytes": 2364519, + "sha256": "1d3ca33c2d3fa2f80dd020a52e2f9aa734426a255039ddbeb26978294b1b056d", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Ridge Classifier", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__ridge_classifier.pkl", + "source_fold": "random_holdout", + "inner_score": 0.6116, + "params": { + "alpha": 3.64511667400448, + "fit_intercept": false, + "max_iter": 1522, + "tol": 6.319915276046356e-05, + "solver": "sag" + }, + "metrics": { + "accuracy": 0.6491, + "balanced_accuracy": 0.4847, + "macro_f1": 0.4923, + "weighted_f1": 0.6138, + "quadratic_kappa": 0.4321, + "kappa": 0.4152, + "mcc": 0.4286 + }, + "bytes": 5460, + "sha256": "3ec918b7a29e97eb50805c91628d368fdeb6c69fe8e8e67e22ef11b4e9ec072b", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "SGD (hinge)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__sgd_hinge.pkl", + "source_fold": "random_holdout", + "inner_score": 0.6321, + "params": { + "penalty": "l2", + "alpha": 2.2515499866245612e-06, + "l1_ratio": 0.7576317076071589, + "fit_intercept": true, + "max_iter": 1710, + "tol": 1.6321601585180913e-05, + "learning_rate": "adaptive", + "eta0": 0.6752718370815053, + "early_stopping": false, + "validation_fraction": 0.08311056948401674, + "n_iter_no_change": 4 + }, + "metrics": { + "accuracy": 0.6736, + "balanced_accuracy": 0.5917, + "macro_f1": 0.5715, + "weighted_f1": 0.6606, + "quadratic_kappa": 0.5927, + "kappa": 0.4869, + "mcc": 0.4917 + }, + "bytes": 5438, + "sha256": "52d0ab3869aab543e325f80253801b31222adda8bf462cad90f81aff6c9a5a62", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "SVM (RBF)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__svm_rbf.pkl", + "source_fold": "random_holdout", + "inner_score": 0.687, + "params": { + "C": 87.00416272425284, + "gamma": "scale", + "tol": 0.0008591574935377383, + "shrinking": false, + "class_weight": null, + "decision_function_shape": "ovr" + }, + "metrics": { + "accuracy": 0.6906, + "balanced_accuracy": 0.6336, + "macro_f1": 0.6199, + "weighted_f1": 0.6908, + "quadratic_kappa": 0.6361, + "kappa": 0.5276, + "mcc": 0.5293, + "roc_auc_weighted_ovr": 0.8629 + }, + "bytes": 68766, + "sha256": "11f12577a98883c69cb233a01f3682a2e3846e28c2076e34002cd1293d92c9b1", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "SVM (polynomial)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__svm_polynomial.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7232, + "params": { + "C": 86.06977879918506, + "gamma": "scale", + "tol": 0.00012235878835636325, + "shrinking": false, + "class_weight": null, + "decision_function_shape": "ovr", + "degree": 5, + "coef0": 0.5911404803031264 + }, + "metrics": { + "accuracy": 0.7302, + "balanced_accuracy": 0.6844, + "macro_f1": 0.67, + "weighted_f1": 0.7324, + "quadratic_kappa": 0.6756, + "kappa": 0.589, + "mcc": 0.5895, + "roc_auc_weighted_ovr": 0.881 + }, + "bytes": 67092, + "sha256": "e1c73dda1047c5a30d2849fff8b56d610e76c04d4f4435bf3bdc294b3c536720", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Soft Voting (RF+XGB+LR)", + "protocol": "random", + "selection": "composed", + "file": "classifiers/random/printability__soft_voting_rf_xgb_lr.pkl", + "source_fold": "random_holdout", + "inner_score": null, + "params": { + "composed_from": [ + "Logistic Regression", + "Random Forest", + "XGBoost" + ] + }, + "metrics": { + "accuracy": 0.7604, + "balanced_accuracy": 0.6936, + "macro_f1": 0.7027, + "weighted_f1": 0.7562, + "quadratic_kappa": 0.7042, + "kappa": 0.6226, + "mcc": 0.6249, + "roc_auc_weighted_ovr": 0.9003 + }, + "bytes": 6042980, + "sha256": "157f10b8830b617c7ceaddde35d625713c2dd4e0dfef8066a2ebfde00560f578", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Stacking (RF+XGB+LR -> LR)", + "protocol": "random", + "selection": "composed", + "file": "classifiers/random/printability__stacking_rf_xgb_lr_lr.pkl", + "source_fold": "random_holdout", + "inner_score": null, + "params": { + "composed_from": [ + "Logistic Regression", + "Random Forest", + "XGBoost" + ] + }, + "metrics": { + "accuracy": 0.7755, + "balanced_accuracy": 0.7187, + "macro_f1": 0.7297, + "weighted_f1": 0.7731, + "quadratic_kappa": 0.7802, + "kappa": 0.6487, + "mcc": 0.6493, + "roc_auc_weighted_ovr": 0.9159 + }, + "bytes": 6043500, + "sha256": "42d86a8fced6c364b61c8a5bff18beeae49c1e11accb2fffc2c52322c6743db6", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "XGBoost", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__xgboost.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7482, + "params": { + "n_estimators": 289, + "max_depth": 9, + "learning_rate": 0.159977753367693, + "gamma": 0.1197890583572232, + "min_child_weight": 5.624222262042716, + "subsample": 0.83878685458189, + "colsample_bytree": 0.6808004828060145, + "reg_alpha": 0.5208355522870847, + "reg_lambda": 0.04085587711752425 + }, + "metrics": { + "accuracy": 0.7623, + "balanced_accuracy": 0.7081, + "macro_f1": 0.7093, + "weighted_f1": 0.7605, + "quadratic_kappa": 0.7784, + "kappa": 0.6306, + "mcc": 0.631, + "roc_auc_weighted_ovr": 0.9142 + }, + "bytes": 651415, + "sha256": "456035d8a5b709323c9025021c4cc405b320ef19c12922d5343d3544f5e31fda", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "k-NN (distance weighted)", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__k_nn_distance_weighted.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7379, + "params": { + "n_neighbors": 9, + "algorithm": "brute", + "leaf_size": 19, + "p": 1, + "metric": "manhattan" + }, + "metrics": { + "accuracy": 0.7491, + "balanced_accuracy": 0.7151, + "macro_f1": 0.7137, + "weighted_f1": 0.7492, + "quadratic_kappa": 0.75, + "kappa": 0.6122, + "mcc": 0.6129, + "roc_auc_weighted_ovr": 0.8495 + }, + "bytes": 53689, + "sha256": "09f59127db7f4e8d30b7ad810bcdd3eda19749fb13b10ac68fe4e14a32991767", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "k-Nearest Neighbours", + "protocol": "random", + "selection": "weighted", + "file": "classifiers/random/printability__k_nearest_neighbours.pkl", + "source_fold": "random_holdout", + "inner_score": 0.7282, + "params": { + "n_neighbors": 1, + "algorithm": "auto", + "leaf_size": 11, + "p": 1, + "metric": "minkowski" + }, + "metrics": { + "accuracy": 0.7358, + "balanced_accuracy": 0.7028, + "macro_f1": 0.7067, + "weighted_f1": 0.7352, + "quadratic_kappa": 0.7489, + "kappa": 0.5903, + "mcc": 0.5905, + "roc_auc_weighted_ovr": 0.797 + }, + "bytes": 53695, + "sha256": "454d01feb6c9b0927614bf2537641b445aac81664bb0df17b33a7472fccceb49", + "reused": true + }, + { + "family": "dl", + "task": "cell_response", + "model": "1D_CNN", + "protocol": "random", + "selection": "weighted", + "file": "deep/random/cell_response__1d_cnn.pt", + "source_fold": "random_holdout", + "inner_score": 0.8076, + "params": { + "n_layers": 6, + "hidden_dim": 192, + "lr": 0.004709568232612032, + "weight_decay": 0.0030186778322337827, + "dropout": 0.24554495469106954, + "batch_size": 32, + "activation": "GELU", + "optimizer": "AdamW", + "max_epochs": 210, + "class_weight": null + }, + "metrics": { + "accuracy": 0.7208, + "balanced_accuracy": 0.409, + "macro_f1": 0.4026, + "weighted_f1": 0.689, + "quadratic_kappa": 0.7004, + "kappa": 0.5113, + "mcc": 0.5172, + "roc_auc_weighted_ovr": 0.9178 + }, + "bytes": 2255387, + "sha256": "3c8f12a5c725035f14226225d60bda49a4340889d2d5d7701e81f0fece22c60f", + "reused": true + }, + { + "family": "dl", + "task": "cell_response", + "model": "FT_Transformer", + "protocol": "random", + "selection": "weighted", + "file": "deep/random/cell_response__ft_transformer.pt", + "source_fold": "random_holdout", + "inner_score": 0.7965, + "params": { + "n_layers": 3, + "hidden_dim": 320, + "lr": 0.00011600050472444694, + "weight_decay": 0.0008107184270673576, + "dropout": 0.17217609657257732, + "batch_size": 64, + "activation": "SiLU", + "optimizer": "Adam", + "max_epochs": 282, + "class_weight": null + }, + "metrics": { + "accuracy": 0.7585, + "balanced_accuracy": 0.4926, + "macro_f1": 0.5026, + "weighted_f1": 0.754, + "quadratic_kappa": 0.8259, + "kappa": 0.5974, + "mcc": 0.6001, + "roc_auc_weighted_ovr": 0.9514 + }, + "bytes": 10286706, + "sha256": "5ae43a34114cd48bd3e3c699cbd5a4f008554a9f0ff4b29c085786a248a617bc", + "reused": true + }, + { + "family": "dl", + "task": "cell_response", + "model": "MLP", + "protocol": "random", + "selection": "weighted", + "file": "deep/random/cell_response__mlp.pt", + "source_fold": "random_holdout", + "inner_score": 0.7853, + "params": { + "n_layers": 3, + "hidden_dim": 512, + "lr": 0.0004466908989057944, + "weight_decay": 0.0006688035071110994, + "dropout": 0.10450128119701933, + "batch_size": 32, + "activation": "GELU", + "optimizer": "AdamW", + "max_epochs": 339, + "class_weight": null + }, + "metrics": { + "accuracy": 0.7226, + "balanced_accuracy": 0.4451, + "macro_f1": 0.4471, + "weighted_f1": 0.7092, + "quadratic_kappa": 0.7326, + "kappa": 0.529, + "mcc": 0.5308, + "roc_auc_weighted_ovr": 0.9334 + }, + "bytes": 2459549, + "sha256": "f89dec0eab3df20f4030b36d6d270d27578b3eeb5959638f00cc0f13094fa8ad", + "reused": true + }, + { + "family": "dl", + "task": "cell_response", + "model": "NODE_Lite", + "protocol": "random", + "selection": "weighted", + "file": "deep/random/cell_response__node_lite.pt", + "source_fold": "random_holdout", + "inner_score": 0.7863, + "params": { + "n_layers": 5, + "hidden_dim": 288, + "lr": 0.007752946303449436, + "weight_decay": 1.0581114308038657e-05, + "dropout": 0.29416310309183125, + "batch_size": 256, + "activation": "GELU", + "optimizer": "AdamW", + "max_epochs": 271, + "class_weight": "balanced" + }, + "metrics": { + "accuracy": 0.7189, + "balanced_accuracy": 0.4225, + "macro_f1": 0.4211, + "weighted_f1": 0.69, + "quadratic_kappa": 0.6882, + "kappa": 0.5052, + "mcc": 0.5104, + "roc_auc_weighted_ovr": 0.9256 + }, + "bytes": 745231, + "sha256": "037bb2ce02a8e9a2cdf5289e95dc70715a31581df37ae8de8723cb03b83220c3", + "reused": true + }, + { + "family": "dl", + "task": "cell_response", + "model": "ResNet", + "protocol": "random", + "selection": "weighted", + "file": "deep/random/cell_response__resnet.pt", + "source_fold": "random_holdout", + "inner_score": 0.7921, + "params": { + "n_layers": 1, + "hidden_dim": 224, + "lr": 0.0071137895547408935, + "weight_decay": 1.1764249709986261e-05, + "dropout": 0.1393661397532897, + "batch_size": 32, + "activation": "tanh", + "optimizer": "AdamW", + "max_epochs": 144, + "class_weight": null + }, + "metrics": { + "accuracy": 0.7623, + "balanced_accuracy": 0.5313, + "macro_f1": 0.5303, + "weighted_f1": 0.7533, + "quadratic_kappa": 0.786, + "kappa": 0.5983, + "mcc": 0.5999, + "roc_auc_weighted_ovr": 0.9495 + }, + "bytes": 357543, + "sha256": "5035fd746ebd5a80842e547a74ef84b9b6b7839b84f90a387130b0f3f68ec0f9", + "reused": true + }, + { + "family": "dl", + "task": "cell_response", + "model": "TabNet_Lite", + "protocol": "random", + "selection": "weighted", + "file": "deep/random/cell_response__tabnet_lite.pt", + "source_fold": "random_holdout", + "inner_score": 0.7967, + "params": { + "n_layers": 6, + "hidden_dim": 416, + "lr": 0.0002263980232546724, + "weight_decay": 0.00602456409460787, + "dropout": 0.3753007065384213, + "batch_size": 32, + "activation": "ELU", + "optimizer": "Adam", + "max_epochs": 256, + "class_weight": null + }, + "metrics": { + "accuracy": 0.7151, + "balanced_accuracy": 0.463, + "macro_f1": 0.4725, + "weighted_f1": 0.7085, + "quadratic_kappa": 0.7028, + "kappa": 0.5203, + "mcc": 0.5217, + "roc_auc_weighted_ovr": 0.905 + }, + "bytes": 3162629, + "sha256": "f64e7187426e3c8235da619d36604c80f02880b86b973bc701111459818d7a5d", + "reused": true + }, + { + "family": "dl", + "task": "printability", + "model": "1D_CNN", + "protocol": "random", + "selection": "weighted", + "file": "deep/random/printability__1d_cnn.pt", + "source_fold": "random_holdout", + "inner_score": 0.7167, + "params": { + "n_layers": 5, + "hidden_dim": 384, + "lr": 0.004098777940402098, + "weight_decay": 0.00022317484183128537, + "dropout": 0.3307896801523583, + "batch_size": 64, + "activation": "GELU", + "optimizer": "AdamW", + "max_epochs": 224, + "class_weight": null + }, + "metrics": { + "accuracy": 0.6566, + "balanced_accuracy": 0.5884, + "macro_f1": 0.5582, + "weighted_f1": 0.6538, + "quadratic_kappa": 0.5338, + "kappa": 0.4743, + "mcc": 0.4792, + "roc_auc_weighted_ovr": 0.8497 + }, + "bytes": 7139363, + "sha256": "9ef3649953646a55eae781c730477fffe126874fc5737ecd3b923ab350f0a652", + "reused": true + }, + { + "family": "dl", + "task": "printability", + "model": "FT_Transformer", + "protocol": "random", + "selection": "weighted", + "file": "deep/random/printability__ft_transformer.pt", + "source_fold": "random_holdout", + "inner_score": 0.7351, + "params": { + "n_layers": 3, + "hidden_dim": 192, + "lr": 0.00010351308939753292, + "weight_decay": 0.002069805688069605, + "dropout": 0.26799093540873087, + "batch_size": 64, + "activation": "SELU", + "optimizer": "AdamW", + "max_epochs": 318, + "class_weight": null + }, + "metrics": { + "accuracy": 0.7264, + "balanced_accuracy": 0.6849, + "macro_f1": 0.6776, + "weighted_f1": 0.728, + "quadratic_kappa": 0.6707, + "kappa": 0.5815, + "mcc": 0.5825, + "roc_auc_weighted_ovr": 0.8687 + }, + "bytes": 3817667, + "sha256": "2efe5b49d2317a1eb319c72c4608da81c25ea0988f30084607ff02e2e5ca9cc8", + "reused": true + }, + { + "family": "dl", + "task": "printability", + "model": "MLP", + "protocol": "random", + "selection": "weighted", + "file": "deep/random/printability__mlp.pt", + "source_fold": "random_holdout", + "inner_score": 0.7386, + "params": { + "n_layers": 3, + "hidden_dim": 480, + "lr": 0.0007485765201520371, + "weight_decay": 0.00011124061899236904, + "dropout": 0.17010524647081268, + "batch_size": 32, + "activation": "tanh", + "optimizer": "AdamW", + "max_epochs": 227, + "class_weight": "balanced" + }, + "metrics": { + "accuracy": 0.6849, + "balanced_accuracy": 0.6652, + "macro_f1": 0.6308, + "weighted_f1": 0.6938, + "quadratic_kappa": 0.6367, + "kappa": 0.5341, + "mcc": 0.5378, + "roc_auc_weighted_ovr": 0.8671 + }, + "bytes": 2181568, + "sha256": "24e5457124f01955414b7b080793e93f9b6efe1b9ec8adddcc8d2bfd80d5b692", + "reused": true + }, + { + "family": "dl", + "task": "printability", + "model": "NODE_Lite", + "protocol": "random", + "selection": "weighted", + "file": "deep/random/printability__node_lite.pt", + "source_fold": "random_holdout", + "inner_score": 0.7194, + "params": { + "n_layers": 5, + "hidden_dim": 224, + "lr": 0.0074723976893329375, + "weight_decay": 2.5856088907313374e-05, + "dropout": 0.20231990531507757, + "batch_size": 64, + "activation": "tanh", + "optimizer": "AdamW", + "max_epochs": 205, + "class_weight": "balanced" + }, + "metrics": { + "accuracy": 0.6849, + "balanced_accuracy": 0.6638, + "macro_f1": 0.6221, + "weighted_f1": 0.6967, + "quadratic_kappa": 0.64, + "kappa": 0.5376, + "mcc": 0.5426, + "roc_auc_weighted_ovr": 0.8632 + }, + "bytes": 576316, + "sha256": "c6cfab0ad5fb11fb0974e8b6c65e36fbfa099c13d0e665e5610fa0f3a4aa4bce", + "reused": true + }, + { + "family": "dl", + "task": "printability", + "model": "ResNet", + "protocol": "random", + "selection": "weighted", + "file": "deep/random/printability__resnet.pt", + "source_fold": "random_holdout", + "inner_score": 0.7632, + "params": { + "n_layers": 6, + "hidden_dim": 160, + "lr": 0.0022093834415066287, + "weight_decay": 0.0019268985325226215, + "dropout": 0.17129126319771992, + "batch_size": 32, + "activation": "GELU", + "optimizer": "AdamW", + "max_epochs": 168, + "class_weight": null + }, + "metrics": { + "accuracy": 0.7396, + "balanced_accuracy": 0.6957, + "macro_f1": 0.6903, + "weighted_f1": 0.7406, + "quadratic_kappa": 0.6685, + "kappa": 0.6022, + "mcc": 0.6036, + "roc_auc_weighted_ovr": 0.8765 + }, + "bytes": 753903, + "sha256": "3d4f4f7faa7b5a231e834c8a691601c8e80bb5694adbba815f27be1653ba5b05", + "reused": true + }, + { + "family": "dl", + "task": "printability", + "model": "TabNet_Lite", + "protocol": "random", + "selection": "weighted", + "file": "deep/random/printability__tabnet_lite.pt", + "source_fold": "random_holdout", + "inner_score": 0.7658, + "params": { + "n_layers": 4, + "hidden_dim": 288, + "lr": 0.003059113634053077, + "weight_decay": 0.00152153896959905, + "dropout": 0.22295374856479444, + "batch_size": 64, + "activation": "GELU", + "optimizer": "Adam", + "max_epochs": 286, + "class_weight": null + }, + "metrics": { + "accuracy": 0.7208, + "balanced_accuracy": 0.6659, + "macro_f1": 0.6555, + "weighted_f1": 0.7207, + "quadratic_kappa": 0.6541, + "kappa": 0.5704, + "mcc": 0.5728, + "roc_auc_weighted_ovr": 0.8648 + }, + "bytes": 1472384, + "sha256": "65e43d38242eea53dd7709d714d052368c13984a1dfb1bb1c1b7e6c2a55f7c42", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "AdaBoost", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__adaboost.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7174, + "params": { + "n_estimators": 114, + "learning_rate": 1.1118095995028556 + }, + "metrics": { + "accuracy": 0.7439, + "balanced_accuracy": 0.3812, + "macro_f1": 0.3713, + "weighted_f1": 0.7065, + "quadratic_kappa": 0.6718, + "kappa": 0.5168, + "mcc": 0.5255, + "roc_auc_weighted_ovr": 0.9209 + }, + "bytes": 22910, + "sha256": "6718b55ac9002c567ead988ac4122e5efd5a93b159c6cfa6a05e9deb86f0c2d4", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Bagged Trees", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__bagged_trees.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7321, + "params": { + "n_estimators": 164, + "max_samples": 0.8465207974838976, + "max_features": 0.5236732743909801, + "bootstrap": true, + "estimator__max_depth": 6, + "estimator__min_samples_leaf": 5 + }, + "metrics": { + "accuracy": 0.7927, + "balanced_accuracy": 0.4018, + "macro_f1": 0.3678, + "weighted_f1": 0.7505, + "quadratic_kappa": 0.8452, + "kappa": 0.6244, + "mcc": 0.6443, + "roc_auc_weighted_ovr": 0.9296 + }, + "bytes": 382728, + "sha256": "ea64fd33edab665f7b1d506a26dabf4a14dc1489cb0f9306673f4913f488a5d9", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Bernoulli Naive Bayes", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__bernoulli_naive_bayes.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7463, + "params": { + "alpha": 1.1463059609830264, + "binarize": 0.4415823418588022, + "fit_prior": true + }, + "metrics": { + "accuracy": 0.6481, + "balanced_accuracy": 0.3105, + "macro_f1": 0.2974, + "weighted_f1": 0.6178, + "quadratic_kappa": 0.5896, + "kappa": 0.3433, + "mcc": 0.3522, + "roc_auc_weighted_ovr": 0.8892 + }, + "bytes": 3258, + "sha256": "ecec1c0860c02a066cc8ac05df680dc1b1f3212fee7f03becf85cef41a4f92c5", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "CatBoost", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__catboost.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7279, + "params": { + "iterations": 421, + "learning_rate": 0.032989969655395635, + "depth": 3, + "l2_leaf_reg": 1.345164254063931, + "bagging_temperature": 0.5622089857656744, + "random_strength": 0.4793067180743533, + "colsample_bylevel": 0.7499628787296093, + "bootstrap_type": "Bayesian" + }, + "metrics": { + "accuracy": 0.7909, + "balanced_accuracy": 0.4062, + "macro_f1": 0.3784, + "weighted_f1": 0.7544, + "quadratic_kappa": 0.8406, + "kappa": 0.6223, + "mcc": 0.6383, + "roc_auc_weighted_ovr": 0.934 + }, + "bytes": 121222, + "sha256": "267c4d295077c0d39cb88e1fe210e89532dba5d70ab8542d0d7245f52d97ecb2", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Decision Tree", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__decision_tree.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7242, + "params": { + "criterion": "log_loss", + "splitter": "random", + "max_depth": 6, + "min_samples_split": 19, + "min_samples_leaf": 2, + "max_features": null, + "ccp_alpha": 0.007283697002116443 + }, + "metrics": { + "accuracy": 0.7753, + "balanced_accuracy": 0.4062, + "macro_f1": 0.3805, + "weighted_f1": 0.7474, + "quadratic_kappa": 0.8116, + "kappa": 0.5929, + "mcc": 0.598, + "roc_auc_weighted_ovr": 0.8989 + }, + "bytes": 2175, + "sha256": "cef3e394022818f3dddf0c669042ce31ddf3ada27a33dd70ec0855420e810e91", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Extra Tree", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__extra_tree.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.717, + "params": { + "criterion": "log_loss", + "max_depth": 29, + "min_samples_split": 13, + "min_samples_leaf": 4, + "max_features": null, + "ccp_alpha": 0.008232729123756432 + }, + "metrics": { + "accuracy": 0.7718, + "balanced_accuracy": 0.4051, + "macro_f1": 0.3765, + "weighted_f1": 0.7506, + "quadratic_kappa": 0.8342, + "kappa": 0.5939, + "mcc": 0.599, + "roc_auc_weighted_ovr": 0.9147 + }, + "bytes": 2310, + "sha256": "7135c7910a0065842048b019adae927bb3babcc49fbb4056f672e225fae8014e", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Extra Trees", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__extra_trees.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7295, + "params": { + "n_estimators": 167, + "criterion": "entropy", + "max_depth": 55, + "min_samples_split": 18, + "min_samples_leaf": 4, + "max_features": null, + "bootstrap": true, + "ccp_alpha": 0.0031123420497667087 + }, + "metrics": { + "accuracy": 0.7108, + "balanced_accuracy": 0.3146, + "macro_f1": 0.3019, + "weighted_f1": 0.6561, + "quadratic_kappa": 0.5893, + "kappa": 0.4227, + "mcc": 0.4383, + "roc_auc_weighted_ovr": 0.9174 + }, + "bytes": 684133, + "sha256": "a70d107aa98287c1ad1e6db73964474bd4fff09169c048d0febff6afa89103b9", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Gaussian Naive Bayes", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__gaussian_naive_bayes.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.1978, + "params": { + "var_smoothing": 1.767016940294792e-09 + }, + "metrics": { + "accuracy": 0.2038, + "balanced_accuracy": 0.2239, + "macro_f1": 0.1158, + "weighted_f1": 0.2497, + "quadratic_kappa": -0.1297, + "kappa": 0.0033, + "mcc": 0.0042, + "roc_auc_weighted_ovr": 0.6395 + }, + "bytes": 7344, + "sha256": "23630e99b3a4f7d0a488561de7da1b50fdc883fd2bbdc3ec5fbef5b053518a38", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Gradient Boosting", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__gradient_boosting.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.713, + "params": { + "learning_rate": 0.05488306594586391, + "n_estimators": 154, + "subsample": 0.5315656901881007, + "criterion": "squared_error", + "min_samples_split": 13, + "min_samples_leaf": 4, + "max_depth": 3, + "max_features": null, + "ccp_alpha": 1.3700690549896082e-05 + }, + "metrics": { + "accuracy": 0.77, + "balanced_accuracy": 0.3844, + "macro_f1": 0.3615, + "weighted_f1": 0.7428, + "quadratic_kappa": 0.8239, + "kappa": 0.5868, + "mcc": 0.5953, + "roc_auc_weighted_ovr": 0.934 + }, + "bytes": 207376, + "sha256": "50f8eb4225619001edf0f2f0aa30a8a2db05420aacfe947cd17e3e5a12ed4c0b", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Hist Gradient Boosting", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__hist_gradient_boosting.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7231, + "params": { + "learning_rate": 0.011865151832183595, + "max_iter": 163, + "max_leaf_nodes": 27, + "max_depth": 25, + "min_samples_leaf": 24, + "l2_regularization": 0.9643179894952475, + "max_bins": 196, + "early_stopping": true, + "n_iter_no_change": 15 + }, + "metrics": { + "accuracy": 0.7666, + "balanced_accuracy": 0.3948, + "macro_f1": 0.3887, + "weighted_f1": 0.7508, + "quadratic_kappa": 0.8318, + "kappa": 0.5832, + "mcc": 0.5861, + "roc_auc_weighted_ovr": 0.9212 + }, + "bytes": 737332, + "sha256": "a4c033e5ed3503f307e0e6ecf6fc937bdd1475eae282e141fa83cbccc2d622a8", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "LightGBM", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__lightgbm.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7318, + "params": { + "boosting_type": "gbdt", + "num_leaves": 227, + "max_depth": 41, + "learning_rate": 0.010732966845902341, + "n_estimators": 452, + "min_child_weight": 5.393883076914592, + "min_child_samples": 42, + "subsample": 0.9480456499617467, + "subsample_freq": 3, + "colsample_bytree": 0.5550259622638384, + "reg_alpha": 2.279351625419417, + "reg_lambda": 4.271077886262563 + }, + "metrics": { + "accuracy": 0.7805, + "balanced_accuracy": 0.4006, + "macro_f1": 0.3799, + "weighted_f1": 0.7534, + "quadratic_kappa": 0.844, + "kappa": 0.6066, + "mcc": 0.6158, + "roc_auc_weighted_ovr": 0.925 + }, + "bytes": 1108048, + "sha256": "9d0edf6f384346206f78d7d4cdfd34b758bfd2beed18e50321b29a2a815aaf10", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Linear Discriminant", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__linear_discriminant.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7187, + "params": { + "shrinkage": 0.1, + "n_components": 10, + "tol": 0.00788671412999049 + }, + "metrics": { + "accuracy": 0.6132, + "balanced_accuracy": 0.2755, + "macro_f1": 0.2659, + "weighted_f1": 0.5827, + "quadratic_kappa": 0.4364, + "kappa": 0.2659, + "mcc": 0.2707, + "roc_auc_weighted_ovr": 0.8455 + }, + "bytes": 145278, + "sha256": "195c7dcdf5dbe59728f7befc0250bbde6d5ebec2ccf5c824a16a49c336fc4a40", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Linear SVM", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__linear_svm.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.722, + "params": { + "loss": "squared_hinge", + "tol": 0.00029696311170881914, + "C": 0.053552055285881764, + "fit_intercept": false, + "intercept_scaling": 0.20937657556695136, + "max_iter": 4806, + "class_weight": "balanced" + }, + "metrics": { + "accuracy": 0.6167, + "balanced_accuracy": 0.2866, + "macro_f1": 0.2867, + "weighted_f1": 0.5938, + "quadratic_kappa": 0.4949, + "kappa": 0.284, + "mcc": 0.2877 + }, + "bytes": 6434, + "sha256": "192ebe26692066d4e3367057136c4c7a9cbeb5fb21e7037d3293526d81f00683", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Logistic Regression", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__logistic_regression.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7232, + "params": { + "penalty": "l2", + "tol": 0.005564258120476742, + "C": 88.6077480618815, + "fit_intercept": false, + "solver": "newton-cg", + "max_iter": 1110 + }, + "metrics": { + "accuracy": 0.6132, + "balanced_accuracy": 0.2747, + "macro_f1": 0.2679, + "weighted_f1": 0.5739, + "quadratic_kappa": 0.3651, + "kappa": 0.2445, + "mcc": 0.2526, + "roc_auc_weighted_ovr": 0.8549 + }, + "bytes": 6499, + "sha256": "7c0ebbb4d081568119ccae0f79501f201124175b55e377d8e72c3889b34d95ec", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Logistic Regression (balanced)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__logistic_regression_balanced.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7212, + "params": { + "penalty": null, + "tol": 0.0035869350871417225, + "C": 1.5566871395196396, + "fit_intercept": false, + "solver": "lbfgs", + "max_iter": 1426 + }, + "metrics": { + "accuracy": 0.6045, + "balanced_accuracy": 0.3021, + "macro_f1": 0.3021, + "weighted_f1": 0.613, + "quadratic_kappa": 0.5986, + "kappa": 0.3189, + "mcc": 0.321, + "roc_auc_weighted_ovr": 0.8334 + }, + "bytes": 6535, + "sha256": "63baa05d194228bad73304348e43ea4ef41af56a6f4765fcd4302ba26c8beec2", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "MLP (256-128)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__mlp_256_128.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.733, + "params": { + "hidden_layer_sizes": "64", + "activation": "logistic", + "solver": "sgd", + "alpha": 0.09315867831843434, + "learning_rate": "constant", + "learning_rate_init": 0.05439034985544032, + "max_iter": 278, + "tol": 4.974931918853955e-06, + "early_stopping": false + }, + "metrics": { + "accuracy": 0.6307, + "balanced_accuracy": 0.2892, + "macro_f1": 0.2853, + "weighted_f1": 0.5897, + "quadratic_kappa": 0.3338, + "kappa": 0.2723, + "mcc": 0.2801, + "roc_auc_weighted_ovr": 0.8725 + }, + "bytes": 240967, + "sha256": "bc9b8b42ee6ad2937743c882e226ebbd9bf3d047764ce57908fae3ecf8cb8644", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Nearest Centroid", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__nearest_centroid.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.6761, + "params": { + "metric": "euclidean", + "shrink_threshold": 0.3463462348151025 + }, + "metrics": { + "accuracy": 0.6568, + "balanced_accuracy": 0.3415, + "macro_f1": 0.3018, + "weighted_f1": 0.5976, + "quadratic_kappa": 0.5054, + "kappa": 0.3489, + "mcc": 0.3724, + "roc_auc_weighted_ovr": 0.8479 + }, + "bytes": 10569, + "sha256": "e776e38fdb8d7aee6eb5d947f2dedaa2f20553d4f84caa934884b9b8a09d0075", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Passive Aggressive", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__passive_aggressive.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7285, + "params": { + "C": 0.011167353585346007, + "fit_intercept": true, + "max_iter": 1015, + "tol": 4.562214920100225e-05, + "early_stopping": false, + "validation_fraction": 0.22818957756882569, + "n_iter_no_change": 7, + "shuffle": true, + "loss": "hinge", + "class_weight": "balanced", + "average": true + }, + "metrics": { + "accuracy": 0.615, + "balanced_accuracy": 0.2802, + "macro_f1": 0.2839, + "weighted_f1": 0.5769, + "quadratic_kappa": 0.283, + "kappa": 0.2422, + "mcc": 0.2491 + }, + "bytes": 12311, + "sha256": "6a81d84328db5be9d5915b22af9428c54e21f8c223b43b4f4615b4d20c3e8ccf", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Perceptron", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__perceptron.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.6938, + "params": { + "penalty": null, + "alpha": 1.3788198614349076e-07, + "l1_ratio": 0.8604135745095509, + "fit_intercept": false, + "max_iter": 1524, + "tol": 0.00043629529299795517, + "shuffle": true, + "eta0": 1.2120210591905518, + "early_stopping": false, + "validation_fraction": 0.27658910543474596, + "n_iter_no_change": 1, + "class_weight": null + }, + "metrics": { + "accuracy": 0.6847, + "balanced_accuracy": 0.2885, + "macro_f1": 0.2701, + "weighted_f1": 0.6397, + "quadratic_kappa": 0.5541, + "kappa": 0.3934, + "mcc": 0.4045 + }, + "bytes": 3872, + "sha256": "7f7db434cabe8b48152863d515bd922bf801460e466240e48a20c45939297efe", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Quadratic Discriminant", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__quadratic_discriminant.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7279, + "params": { + "pca__n_components": 10, + "reg_param": 0.025769677431349825, + "store_covariance": true, + "tol": 0.0009768137391168961 + }, + "metrics": { + "accuracy": 0.6603, + "balanced_accuracy": 0.3805, + "macro_f1": 0.3717, + "weighted_f1": 0.6877, + "quadratic_kappa": 0.7922, + "kappa": 0.4414, + "mcc": 0.4458, + "roc_auc_weighted_ovr": 0.8906 + }, + "bytes": 21301, + "sha256": "37d356139cc35d8d4ed413b81d7ca13bef0a1e7111da9a2878e3899ff80422fb", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Random Forest", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__random_forest.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7128, + "params": { + "n_estimators": 400, + "criterion": "entropy", + "max_depth": 18, + "min_samples_split": 3, + "min_samples_leaf": 1, + "min_weight_fraction_leaf": 0.08163400564338802, + "max_features": null, + "bootstrap": true, + "ccp_alpha": 0.027521175215396168, + "max_samples": 0.9 + }, + "metrics": { + "accuracy": 0.8014, + "balanced_accuracy": 0.4303, + "macro_f1": 0.3995, + "weighted_f1": 0.7691, + "quadratic_kappa": 0.851, + "kappa": 0.6431, + "mcc": 0.6537, + "roc_auc_weighted_ovr": 0.9381 + }, + "bytes": 81920, + "sha256": "e9bd8366e34973589a25f94db606e93fc76661aa79c9fb1bef92c7f9a4f209cd", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Random Forest (balanced)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__random_forest_balanced.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7333, + "params": { + "n_estimators": 350, + "criterion": "entropy", + "max_depth": 35, + "min_samples_split": 7, + "min_samples_leaf": 1, + "min_weight_fraction_leaf": 0.001254715712481154, + "max_features": "sqrt", + "bootstrap": true, + "ccp_alpha": 0.003503213854927241, + "max_samples": 0.8 + }, + "metrics": { + "accuracy": 0.7857, + "balanced_accuracy": 0.4302, + "macro_f1": 0.4245, + "weighted_f1": 0.7668, + "quadratic_kappa": 0.8381, + "kappa": 0.6175, + "mcc": 0.6234, + "roc_auc_weighted_ovr": 0.9229 + }, + "bytes": 3535623, + "sha256": "bb9c43bba7845f687ba285f2ff1e4725813151b71180441daf9be6eab41ea310", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Ridge Classifier", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__ridge_classifier.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.6868, + "params": { + "alpha": 0.16830448420647048, + "fit_intercept": false, + "max_iter": 436, + "tol": 0.00439266117717107, + "solver": "sparse_cg" + }, + "metrics": { + "accuracy": 0.6098, + "balanced_accuracy": 0.2647, + "macro_f1": 0.2549, + "weighted_f1": 0.5645, + "quadratic_kappa": 0.3405, + "kappa": 0.2209, + "mcc": 0.2307 + }, + "bytes": 6533, + "sha256": "060a133f4aaee743437d6e4aa5f025b2ae8873a9a68c723f4c40b864cce6b250", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "SGD (hinge)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__sgd_hinge.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.713, + "params": { + "penalty": "l1", + "alpha": 0.0007653648178811599, + "l1_ratio": 0.5177276835817703, + "fit_intercept": false, + "max_iter": 1426, + "tol": 0.00242346504306506, + "learning_rate": "adaptive", + "eta0": 0.13306525358361757, + "early_stopping": true, + "validation_fraction": 0.10345733844260593, + "n_iter_no_change": 9 + }, + "metrics": { + "accuracy": 0.7021, + "balanced_accuracy": 0.3234, + "macro_f1": 0.3317, + "weighted_f1": 0.6494, + "quadratic_kappa": 0.5366, + "kappa": 0.3892, + "mcc": 0.4119 + }, + "bytes": 2368, + "sha256": "c0bc2affa43a6360cb1828ffee4829c1f431db6fe4e1cc4965c4c36d9765e09a", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "SVM (RBF)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__svm_rbf.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.742, + "params": { + "C": 2.69264691008618, + "gamma": "scale", + "tol": 0.007902619549708232, + "shrinking": true, + "class_weight": null, + "decision_function_shape": "ovo" + }, + "metrics": { + "accuracy": 0.6446, + "balanced_accuracy": 0.3607, + "macro_f1": 0.3299, + "weighted_f1": 0.6381, + "quadratic_kappa": 0.5079, + "kappa": 0.4015, + "mcc": 0.4182, + "roc_auc_weighted_ovr": 0.826 + }, + "bytes": 67710, + "sha256": "7a56cb352999a9427d8329cbda9fb816680600ffce9ce8741b9109032dc01d00", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "SVM (polynomial)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__svm_polynomial.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7336, + "params": { + "C": 1.285048646530307, + "gamma": "scale", + "tol": 0.0016675962972338468, + "shrinking": true, + "class_weight": null, + "decision_function_shape": "ovr", + "degree": 2, + "coef0": 0.5874142617948831 + }, + "metrics": { + "accuracy": 0.7265, + "balanced_accuracy": 0.3436, + "macro_f1": 0.333, + "weighted_f1": 0.6788, + "quadratic_kappa": 0.6426, + "kappa": 0.462, + "mcc": 0.4745, + "roc_auc_weighted_ovr": 0.9331 + }, + "bytes": 60069, + "sha256": "c50b47e260cd1c664dda374ec80d9ff668cc62133c0b368ddb236cc9aba946ff", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Soft Voting (RF+XGB+LR)", + "protocol": "doi", + "selection": "composed", + "file": "classifiers/doi/cell_response__soft_voting_rf_xgb_lr.pkl", + "source_fold": "doi_holdout", + "inner_score": null, + "params": { + "composed_from": [ + "Logistic Regression", + "Random Forest", + "XGBoost" + ] + }, + "metrics": { + "accuracy": 0.777, + "balanced_accuracy": 0.3831, + "macro_f1": 0.3582, + "weighted_f1": 0.7296, + "quadratic_kappa": 0.7862, + "kappa": 0.5792, + "mcc": 0.5915, + "roc_auc_weighted_ovr": 0.925 + }, + "bytes": 5233166, + "sha256": "c8e3ae78518aa32eb858160556780da44b03dc67908a10451c982fd65cce4b14", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "Stacking (RF+XGB+LR -> LR)", + "protocol": "doi", + "selection": "composed", + "file": "classifiers/doi/cell_response__stacking_rf_xgb_lr_lr.pkl", + "source_fold": "doi_holdout", + "inner_score": null, + "params": { + "composed_from": [ + "Logistic Regression", + "Random Forest", + "XGBoost" + ] + }, + "metrics": { + "accuracy": 0.7944, + "balanced_accuracy": 0.3983, + "macro_f1": 0.3646, + "weighted_f1": 0.7476, + "quadratic_kappa": 0.8348, + "kappa": 0.6203, + "mcc": 0.6359, + "roc_auc_weighted_ovr": 0.9259 + }, + "bytes": 5233836, + "sha256": "22b4220df14239c7e42aebc511ec4e182540de964f3443c3360d977861536cfa", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "XGBoost", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__xgboost.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.7295, + "params": { + "n_estimators": 210, + "max_depth": 6, + "learning_rate": 0.2516091128123238, + "gamma": 1.3875949300856911, + "min_child_weight": 3.0361789604162936, + "subsample": 0.5429778885458338, + "colsample_bytree": 0.615265478981264, + "reg_alpha": 3.3505142232585623, + "reg_lambda": 1.8610739664538896 + }, + "metrics": { + "accuracy": 0.7875, + "balanced_accuracy": 0.4102, + "macro_f1": 0.3907, + "weighted_f1": 0.7563, + "quadratic_kappa": 0.838, + "kappa": 0.6176, + "mcc": 0.6301, + "roc_auc_weighted_ovr": 0.933 + }, + "bytes": 53698, + "sha256": "cadcb8c1442dedfa300f009c6b4c7e528cc5082991adf7b0548ca2408aab3c1d", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "k-NN (distance weighted)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__k_nn_distance_weighted.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.6928, + "params": { + "n_neighbors": 11, + "algorithm": "auto", + "leaf_size": 77, + "p": 3, + "metric": "euclidean" + }, + "metrics": { + "accuracy": 0.6463, + "balanced_accuracy": 0.2673, + "macro_f1": 0.2604, + "weighted_f1": 0.5602, + "quadratic_kappa": 0.2347, + "kappa": 0.2176, + "mcc": 0.2588, + "roc_auc_weighted_ovr": 0.7132 + }, + "bytes": 53475, + "sha256": "dcb4c2c85b2380ff5f50900cd4e56b787992f9218edec66cec29ef936e35244c", + "reused": true + }, + { + "family": "ml", + "task": "cell_response", + "model": "k-Nearest Neighbours", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/cell_response__k_nearest_neighbours.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.6785, + "params": { + "n_neighbors": 7, + "algorithm": "kd_tree", + "leaf_size": 98, + "p": 3, + "metric": "manhattan" + }, + "metrics": { + "accuracy": 0.6463, + "balanced_accuracy": 0.2614, + "macro_f1": 0.2524, + "weighted_f1": 0.5594, + "quadratic_kappa": 0.259, + "kappa": 0.2169, + "mcc": 0.258, + "roc_auc_weighted_ovr": 0.7535 + }, + "bytes": 114934, + "sha256": "d48b96126de112c2f87c3fc6a7ba15a088ae26649be6ea8287398fc807073248", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "AdaBoost", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__adaboost.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.5069, + "params": { + "n_estimators": 265, + "learning_rate": 0.09953701633772714 + }, + "metrics": { + "accuracy": 0.5662, + "balanced_accuracy": 0.3893, + "macro_f1": 0.3161, + "weighted_f1": 0.529, + "quadratic_kappa": 0.4157, + "kappa": 0.3297, + "mcc": 0.3875, + "roc_auc_weighted_ovr": 0.7382 + }, + "bytes": 46120, + "sha256": "7a77924a952a29949f0ec6e9e7fe7de18d46b84a79a6c698538eafaf5da9477c", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Bagged Trees", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__bagged_trees.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4768, + "params": { + "n_estimators": 29, + "max_samples": 0.9979389768571496, + "max_features": 0.6861269508008516, + "bootstrap": true, + "estimator__max_depth": 3, + "estimator__min_samples_leaf": 7 + }, + "metrics": { + "accuracy": 0.5575, + "balanced_accuracy": 0.3853, + "macro_f1": 0.312, + "weighted_f1": 0.5217, + "quadratic_kappa": 0.4067, + "kappa": 0.3191, + "mcc": 0.3779, + "roc_auc_weighted_ovr": 0.7481 + }, + "bytes": 23203, + "sha256": "40b8c78b16846ab71b44eae5a56d3ba2998433807fce82e6edabd03238ee85f6", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Bernoulli Naive Bayes", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__bernoulli_naive_bayes.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.5034, + "params": { + "alpha": 7.5873471181147485, + "binarize": 0.04550110514915019, + "fit_prior": true + }, + "metrics": { + "accuracy": 0.5523, + "balanced_accuracy": 0.3473, + "macro_f1": 0.315, + "weighted_f1": 0.5192, + "quadratic_kappa": 0.3927, + "kappa": 0.2662, + "mcc": 0.2749, + "roc_auc_weighted_ovr": 0.7001 + }, + "bytes": 3499, + "sha256": "a3bf4202f8e021bfd4ea6a434500d03be3a691bb764f46a75614fae41bde7b6e", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "CatBoost", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__catboost.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4894, + "params": { + "iterations": 182, + "learning_rate": 0.16137998268604617, + "depth": 4, + "l2_leaf_reg": 3.1118298962854167, + "bagging_temperature": 0.0015529392540532538, + "random_strength": 3.264163026092506, + "colsample_bylevel": 0.994186946913854, + "bootstrap_type": "Bayesian" + }, + "metrics": { + "accuracy": 0.5401, + "balanced_accuracy": 0.3214, + "macro_f1": 0.2979, + "weighted_f1": 0.4923, + "quadratic_kappa": 0.277, + "kappa": 0.2047, + "mcc": 0.2129, + "roc_auc_weighted_ovr": 0.7264 + }, + "bytes": 63112, + "sha256": "0043b33c0a56b401eb218eb194dca4d41b2514282a1f5583b493bdebf4d3120a", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Decision Tree", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__decision_tree.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.5057, + "params": { + "criterion": "gini", + "splitter": "best", + "max_depth": 32, + "min_samples_split": 7, + "min_samples_leaf": 11, + "max_features": null, + "ccp_alpha": 0.037777556927152435 + }, + "metrics": { + "accuracy": 0.5575, + "balanced_accuracy": 0.3853, + "macro_f1": 0.312, + "weighted_f1": 0.5217, + "quadratic_kappa": 0.4067, + "kappa": 0.3191, + "mcc": 0.3779, + "roc_auc_weighted_ovr": 0.7439 + }, + "bytes": 1078, + "sha256": "3e4f1f7d68acc064f9bd93c75ef4b1f919e0d16babdc047be2601e50849ac002", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Extra Tree", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__extra_tree.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4677, + "params": { + "criterion": "log_loss", + "max_depth": 9, + "min_samples_split": 19, + "min_samples_leaf": 11, + "max_features": null, + "ccp_alpha": 0.031782406757632 + }, + "metrics": { + "accuracy": 0.601, + "balanced_accuracy": 0.3872, + "macro_f1": 0.326, + "weighted_f1": 0.5291, + "quadratic_kappa": 0.1874, + "kappa": 0.3158, + "mcc": 0.3359, + "roc_auc_weighted_ovr": 0.7617 + }, + "bytes": 1310, + "sha256": "fe1dbc2a58a675551ef5f91b9abe4890b78bef4cc9626733fff7ad99f9a8bb80", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Extra Trees", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__extra_trees.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.5076, + "params": { + "n_estimators": 331, + "criterion": "gini", + "max_depth": 47, + "min_samples_split": 3, + "min_samples_leaf": 1, + "max_features": null, + "bootstrap": true, + "ccp_alpha": 0.01305999448156043 + }, + "metrics": { + "accuracy": 0.5523, + "balanced_accuracy": 0.3839, + "macro_f1": 0.3098, + "weighted_f1": 0.5172, + "quadratic_kappa": 0.4003, + "kappa": 0.3138, + "mcc": 0.3748, + "roc_auc_weighted_ovr": 0.7496 + }, + "bytes": 103496, + "sha256": "efd2d08153ac73d591c9e078bc441748e7b0aea88b9dc4b7c155c45c45e10bc6", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Gaussian Naive Bayes", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__gaussian_naive_bayes.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.2078, + "params": { + "var_smoothing": 7.48812265223046e-06 + }, + "metrics": { + "accuracy": 0.223, + "balanced_accuracy": 0.2779, + "macro_f1": 0.2233, + "weighted_f1": 0.241, + "quadratic_kappa": 0.0552, + "kappa": 0.0203, + "mcc": 0.0246, + "roc_auc_weighted_ovr": 0.5327 + }, + "bytes": 6137, + "sha256": "4183d69c4b81e2ff38e386a2f0fa1cee1462946b367e129b686fea04d9b2bc67", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Gradient Boosting", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__gradient_boosting.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.5057, + "params": { + "learning_rate": 0.0927163207094897, + "n_estimators": 65, + "subsample": 0.9202948128395309, + "criterion": "friedman_mse", + "min_samples_split": 7, + "min_samples_leaf": 11, + "max_depth": 12, + "max_features": null, + "ccp_alpha": 0.007226700975547 + }, + "metrics": { + "accuracy": 0.5575, + "balanced_accuracy": 0.3853, + "macro_f1": 0.312, + "weighted_f1": 0.5217, + "quadratic_kappa": 0.4067, + "kappa": 0.3191, + "mcc": 0.3779, + "roc_auc_weighted_ovr": 0.7369 + }, + "bytes": 15079, + "sha256": "35c4ec17c771ca1725a3716a166e74da2fd86ca5cf856b49b84ebcf3f9451f11", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Hist Gradient Boosting", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__hist_gradient_boosting.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4758, + "params": { + "learning_rate": 0.01872150773344914, + "max_iter": 403, + "max_leaf_nodes": 14, + "max_depth": 21, + "min_samples_leaf": 25, + "l2_regularization": 0.6991809333892368, + "max_bins": 171, + "early_stopping": false, + "n_iter_no_change": 18 + }, + "metrics": { + "accuracy": 0.5087, + "balanced_accuracy": 0.31, + "macro_f1": 0.3108, + "weighted_f1": 0.4669, + "quadratic_kappa": 0.329, + "kappa": 0.134, + "mcc": 0.1398, + "roc_auc_weighted_ovr": 0.7086 + }, + "bytes": 926648, + "sha256": "aba925654b31f64c58972e00c44be5f6d1fbe62cd3d8c53c5aad7b088b5a769c", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "LightGBM", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__lightgbm.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4882, + "params": { + "boosting_type": "dart", + "num_leaves": 216, + "max_depth": 27, + "learning_rate": 0.15494686217496312, + "n_estimators": 389, + "min_child_weight": 9.246032704766431, + "min_child_samples": 48, + "subsample": 0.7573672206807439, + "subsample_freq": 5, + "colsample_bytree": 0.6498920559104804, + "reg_alpha": 3.5167229267399045, + "reg_lambda": 5.295603666889615 + }, + "metrics": { + "accuracy": 0.4948, + "balanced_accuracy": 0.2914, + "macro_f1": 0.2754, + "weighted_f1": 0.4416, + "quadratic_kappa": 0.2883, + "kappa": 0.1007, + "mcc": 0.1062, + "roc_auc_weighted_ovr": 0.7223 + }, + "bytes": 876702, + "sha256": "bf72cdc2ca5b18281a734a0e21739359d0cb6d763f77432c1b8c347768b116fe", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Linear Discriminant", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__linear_discriminant.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4739, + "params": { + "shrinkage": 0.5, + "n_components": 3, + "tol": 0.000684792009557478 + }, + "metrics": { + "accuracy": 0.4495, + "balanced_accuracy": 0.3263, + "macro_f1": 0.3244, + "weighted_f1": 0.4617, + "quadratic_kappa": 0.4314, + "kappa": 0.1632, + "mcc": 0.1665, + "roc_auc_weighted_ovr": 0.6346 + }, + "bytes": 142358, + "sha256": "496e4da96d3f9abf6c7cd4d88a9d7d33437fecf47e91dcac911bdebecf93f8e3", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Linear SVM", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__linear_svm.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4846, + "params": { + "loss": "squared_hinge", + "tol": 5.342892537395847e-05, + "C": 0.042024885864143345, + "fit_intercept": true, + "intercept_scaling": 2.2738355236928802, + "max_iter": 1405, + "class_weight": "balanced" + }, + "metrics": { + "accuracy": 0.4808, + "balanced_accuracy": 0.3831, + "macro_f1": 0.3716, + "weighted_f1": 0.4876, + "quadratic_kappa": 0.3952, + "kappa": 0.2033, + "mcc": 0.2062 + }, + "bytes": 5349, + "sha256": "b1d6ac4b823fca920139657f1a9ac60e51de98499ce07d8894b5531c6c6b4f7c", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Logistic Regression", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__logistic_regression.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4576, + "params": { + "penalty": "l2", + "tol": 0.0035674973511975678, + "C": 2.60441630415354, + "fit_intercept": false, + "solver": "lbfgs", + "max_iter": 1526 + }, + "metrics": { + "accuracy": 0.4111, + "balanced_accuracy": 0.2227, + "macro_f1": 0.2072, + "weighted_f1": 0.3745, + "quadratic_kappa": 0.1881, + "kappa": -0.0103, + "mcc": -0.0106, + "roc_auc_weighted_ovr": 0.5312 + }, + "bytes": 5349, + "sha256": "8b6ebca01f80d053fe4f58d445d872979bd095d32cab997cede91ce7f636c3b6", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Logistic Regression (balanced)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__logistic_regression_balanced.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4654, + "params": { + "penalty": "l2", + "tol": 0.0004086458374105822, + "C": 0.7075713679745544, + "fit_intercept": false, + "solver": "saga", + "max_iter": 726 + }, + "metrics": { + "accuracy": 0.4094, + "balanced_accuracy": 0.3504, + "macro_f1": 0.3314, + "weighted_f1": 0.4254, + "quadratic_kappa": 0.3412, + "kappa": 0.1311, + "mcc": 0.1349, + "roc_auc_weighted_ovr": 0.5499 + }, + "bytes": 5409, + "sha256": "febcd9de7794eb37e1defb790bcb7c38fdba565d02fa0c422b78b99d57fd7e9a", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "MLP (256-128)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__mlp_256_128.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4747, + "params": { + "hidden_layer_sizes": "128_64", + "activation": "tanh", + "solver": "sgd", + "alpha": 1.518091449100529e-05, + "learning_rate": "invscaling", + "learning_rate_init": 0.07752153512517584, + "max_iter": 831, + "tol": 1.1047078082258689e-06, + "early_stopping": true + }, + "metrics": { + "accuracy": 0.4704, + "balanced_accuracy": 0.3866, + "macro_f1": 0.3824, + "weighted_f1": 0.4852, + "quadratic_kappa": 0.3796, + "kappa": 0.2061, + "mcc": 0.2126, + "roc_auc_weighted_ovr": 0.6448 + }, + "bytes": 436331, + "sha256": "2f54dcee6b5b79afed0e8f06de8e5f780db6810925e3ea7b7b0b811a6686dbb7", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Nearest Centroid", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__nearest_centroid.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4379, + "params": { + "metric": "euclidean", + "shrink_threshold": 0.49371148383200575 + }, + "metrics": { + "accuracy": 0.3275, + "balanced_accuracy": 0.3282, + "macro_f1": 0.2884, + "weighted_f1": 0.3655, + "quadratic_kappa": 0.3072, + "kappa": 0.1197, + "mcc": 0.1347, + "roc_auc_weighted_ovr": 0.6268 + }, + "bytes": 8358, + "sha256": "cd215a76d3a897dfad9e4a9602630ac2e447174d82bcc26914e408eb375a13f7", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Passive Aggressive", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__passive_aggressive.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4889, + "params": { + "C": 4667.462349328399, + "fit_intercept": true, + "max_iter": 1617, + "tol": 0.0002698802952120931, + "early_stopping": true, + "validation_fraction": 0.0777267950828245, + "n_iter_no_change": 1, + "shuffle": true, + "loss": "hinge", + "class_weight": "balanced", + "average": true + }, + "metrics": { + "accuracy": 0.4094, + "balanced_accuracy": 0.2502, + "macro_f1": 0.2527, + "weighted_f1": 0.3989, + "quadratic_kappa": 0.1913, + "kappa": 0.0333, + "mcc": 0.0339 + }, + "bytes": 10098, + "sha256": "b0bfdb195a43b2eb02c0d5a2f1ec00b6cc74c8d4d54c07021cbbad72db0ca9ac", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Perceptron", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__perceptron.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4386, + "params": { + "penalty": null, + "alpha": 1.608166928363465e-07, + "l1_ratio": 0.9093204020787821, + "fit_intercept": false, + "max_iter": 1004, + "tol": 0.0012030178871154666, + "shuffle": true, + "eta0": 7.556810141274429, + "early_stopping": false, + "validation_fraction": 0.2737068376069122, + "n_iter_no_change": 6, + "class_weight": null + }, + "metrics": { + "accuracy": 0.5122, + "balanced_accuracy": 0.2451, + "macro_f1": 0.1898, + "weighted_f1": 0.3871, + "quadratic_kappa": 0.0872, + "kappa": 0.0062, + "mcc": 0.0094 + }, + "bytes": 3790, + "sha256": "d24111b8b0295eebdd59a9e2ac879c46ea9c368d7ca322b4859c34e1558a6f2c", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Quadratic Discriminant", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__quadratic_discriminant.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4987, + "params": { + "pca__n_components": 13, + "reg_param": 0.56021316175996, + "store_covariance": false, + "tol": 0.0003433487371306303 + }, + "metrics": { + "accuracy": 0.547, + "balanced_accuracy": 0.3458, + "macro_f1": 0.2919, + "weighted_f1": 0.4935, + "quadratic_kappa": 0.2891, + "kappa": 0.2446, + "mcc": 0.2627, + "roc_auc_weighted_ovr": 0.7285 + }, + "bytes": 23376, + "sha256": "7900d9df6f932ba74aa1a8e47316a9a8cab1217ba046e14c7c1fed0622e43ee0", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Random Forest", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__random_forest.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.5301, + "params": { + "n_estimators": 200, + "criterion": "log_loss", + "max_depth": 13, + "min_samples_split": 6, + "min_samples_leaf": 6, + "min_weight_fraction_leaf": 0.18869030979403617, + "max_features": null, + "bootstrap": false, + "ccp_alpha": 0.0061748614666293205, + "max_samples": 0.8 + }, + "metrics": { + "accuracy": 0.5348, + "balanced_accuracy": 0.3872, + "macro_f1": 0.3557, + "weighted_f1": 0.5403, + "quadratic_kappa": 0.3637, + "kappa": 0.3017, + "mcc": 0.3202, + "roc_auc_weighted_ovr": 0.7257 + }, + "bytes": 4532, + "sha256": "1e7d47bfecacf241774aff763cbec7e09e8ec3340eab16a36302351f74a47452", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Random Forest (balanced)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__random_forest_balanced.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4899, + "params": { + "n_estimators": 100, + "criterion": "entropy", + "max_depth": 37, + "min_samples_split": 5, + "min_samples_leaf": 10, + "min_weight_fraction_leaf": 0.06100948759139501, + "max_features": "sqrt", + "bootstrap": true, + "ccp_alpha": 0.05482745624204204, + "max_samples": 0.7 + }, + "metrics": { + "accuracy": 0.5714, + "balanced_accuracy": 0.5239, + "macro_f1": 0.4888, + "weighted_f1": 0.6055, + "quadratic_kappa": 0.5091, + "kappa": 0.3866, + "mcc": 0.4006, + "roc_auc_weighted_ovr": 0.746 + }, + "bytes": 15369, + "sha256": "609212dcb35a64da848914e46ed9ddd38d57f2b9dee38b629bd566878da342dc", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Ridge Classifier", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__ridge_classifier.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.453, + "params": { + "alpha": 0.16830448420647048, + "fit_intercept": false, + "max_iter": 436, + "tol": 0.00439266117717107, + "solver": "sparse_cg" + }, + "metrics": { + "accuracy": 0.4861, + "balanced_accuracy": 0.2645, + "macro_f1": 0.2417, + "weighted_f1": 0.4246, + "quadratic_kappa": 0.2537, + "kappa": 0.0684, + "mcc": 0.0735 + }, + "bytes": 5391, + "sha256": "0fb73e9bee81cf54e33a13fc7a581c404dc9ea05d412e8bd00ca8d406865241b", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "SGD (hinge)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__sgd_hinge.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4722, + "params": { + "penalty": "l2", + "alpha": 4.6146396163090724e-05, + "l1_ratio": 0.3894491613327526, + "fit_intercept": true, + "max_iter": 697, + "tol": 0.0011109430612901588, + "learning_rate": "invscaling", + "eta0": 0.2725124444154003, + "early_stopping": true, + "validation_fraction": 0.2650539661157578, + "n_iter_no_change": 3 + }, + "metrics": { + "accuracy": 0.5052, + "balanced_accuracy": 0.2586, + "macro_f1": 0.224, + "weighted_f1": 0.4116, + "quadratic_kappa": 0.1934, + "kappa": 0.0532, + "mcc": 0.0665 + }, + "bytes": 5129, + "sha256": "5b498b2975c80deca53c322917fa4e29a4d178bfdf21d8df5cc8c8e18cc56738", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "SVM (RBF)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__svm_rbf.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4913, + "params": { + "C": 2.0206264305920913, + "gamma": "scale", + "tol": 0.00010228834000617092, + "shrinking": true, + "class_weight": "balanced", + "decision_function_shape": "ovr" + }, + "metrics": { + "accuracy": 0.4216, + "balanced_accuracy": 0.3155, + "macro_f1": 0.31, + "weighted_f1": 0.4235, + "quadratic_kappa": 0.2179, + "kappa": 0.101, + "mcc": 0.1037, + "roc_auc_weighted_ovr": 0.6178 + }, + "bytes": 70810, + "sha256": "1a26de893dbe4ce1d79dbcdf43270c9c4c66be5a385c9c2577b2fd15ed237647", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "SVM (polynomial)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__svm_polynomial.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4899, + "params": { + "C": 0.013930395693114054, + "gamma": "scale", + "tol": 0.0027672535040671182, + "shrinking": true, + "class_weight": null, + "decision_function_shape": "ovo", + "degree": 3, + "coef0": 0.9805317255080246 + }, + "metrics": { + "accuracy": 0.5732, + "balanced_accuracy": 0.3816, + "macro_f1": 0.3337, + "weighted_f1": 0.5338, + "quadratic_kappa": 0.4026, + "kappa": 0.3119, + "mcc": 0.3269, + "roc_auc_weighted_ovr": 0.7245 + }, + "bytes": 64700, + "sha256": "4360d5904237809e9f44857acb0e33b92db78ff0f95b8749ac89c52324d0d4fe", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Soft Voting (RF+XGB+LR)", + "protocol": "doi", + "selection": "composed", + "file": "classifiers/doi/printability__soft_voting_rf_xgb_lr.pkl", + "source_fold": "doi_holdout", + "inner_score": null, + "params": { + "composed_from": [ + "Logistic Regression", + "Random Forest", + "XGBoost" + ] + }, + "metrics": { + "accuracy": 0.4861, + "balanced_accuracy": 0.2705, + "macro_f1": 0.2529, + "weighted_f1": 0.4449, + "quadratic_kappa": 0.382, + "kappa": 0.115, + "mcc": 0.1181, + "roc_auc_weighted_ovr": 0.7047 + }, + "bytes": 6042977, + "sha256": "b46d562fa35e5eb6b39c8bfdcb9248f099384de348385d957ac8529f78b08605", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "Stacking (RF+XGB+LR -> LR)", + "protocol": "doi", + "selection": "composed", + "file": "classifiers/doi/printability__stacking_rf_xgb_lr_lr.pkl", + "source_fold": "doi_holdout", + "inner_score": null, + "params": { + "composed_from": [ + "Logistic Regression", + "Random Forest", + "XGBoost" + ] + }, + "metrics": { + "accuracy": 0.5366, + "balanced_accuracy": 0.3463, + "macro_f1": 0.2896, + "weighted_f1": 0.4803, + "quadratic_kappa": 0.1446, + "kappa": 0.2265, + "mcc": 0.2431, + "roc_auc_weighted_ovr": 0.7475 + }, + "bytes": 6043495, + "sha256": "c54d27d4ecc2300577c78667b3ea5d1c13abe76517ddca841279deeb0de5245f", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "XGBoost", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__xgboost.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.4787, + "params": { + "n_estimators": 468, + "max_depth": 3, + "learning_rate": 0.2925636669623597, + "gamma": 3.7005024190423357, + "min_child_weight": 2.822653000800327, + "subsample": 0.5118183820822848, + "colsample_bytree": 0.5843582204720735, + "reg_alpha": 0.6635717690098635, + "reg_lambda": 9.533061755694554 + }, + "metrics": { + "accuracy": 0.5714, + "balanced_accuracy": 0.3584, + "macro_f1": 0.3403, + "weighted_f1": 0.5202, + "quadratic_kappa": 0.2412, + "kappa": 0.2472, + "mcc": 0.2581, + "roc_auc_weighted_ovr": 0.7467 + }, + "bytes": 70459, + "sha256": "d49d951cff64d17471162366c10f0d17ee0220c41a57dc3e18090b4bb9c580f0", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "k-NN (distance weighted)", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__k_nn_distance_weighted.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.438, + "params": { + "n_neighbors": 39, + "algorithm": "kd_tree", + "leaf_size": 76, + "p": 4, + "metric": "euclidean" + }, + "metrics": { + "accuracy": 0.3554, + "balanced_accuracy": 0.2527, + "macro_f1": 0.2217, + "weighted_f1": 0.3443, + "quadratic_kappa": -0.0109, + "kappa": 0.0197, + "mcc": 0.0206, + "roc_auc_weighted_ovr": 0.603 + }, + "bytes": 119318, + "sha256": "3471e296d0e832cb03cd800c5d4d3c3ac38c46b24e4cb06eaf848b6a4be06354", + "reused": true + }, + { + "family": "ml", + "task": "printability", + "model": "k-Nearest Neighbours", + "protocol": "doi", + "selection": "weighted", + "file": "classifiers/doi/printability__k_nearest_neighbours.pkl", + "source_fold": "doi_holdout", + "inner_score": 0.438, + "params": { + "n_neighbors": 40, + "algorithm": "auto", + "leaf_size": 84, + "p": 5, + "metric": "minkowski" + }, + "metrics": { + "accuracy": 0.3449, + "balanced_accuracy": 0.2635, + "macro_f1": 0.2463, + "weighted_f1": 0.3667, + "quadratic_kappa": 0.0292, + "kappa": 0.0646, + "mcc": 0.0687, + "roc_auc_weighted_ovr": 0.6093 + }, + "bytes": 53696, + "sha256": "84ac69a25645b01f3013259103ee4ab8e3f70b37a49702df451fbb59f55bd0de", + "reused": true + }, + { + "family": "dl", + "task": "cell_response", + "model": "1D_CNN", + "protocol": "doi", + "selection": "weighted", + "file": "deep/doi/cell_response__1d_cnn.pt", + "source_fold": "doi_holdout", + "inner_score": 0.5875, + "params": { + "n_layers": 6, + "hidden_dim": 160, + "lr": 0.0022093834415066287, + "weight_decay": 0.0019268985325226215, + "dropout": 0.17129126319771992, + "batch_size": 32, + "activation": "GELU", + "optimizer": "AdamW", + "max_epochs": 168, + "class_weight": null + }, + "metrics": { + "accuracy": 0.5697, + "balanced_accuracy": 0.3697, + "macro_f1": 0.3404, + "weighted_f1": 0.6198, + "quadratic_kappa": 0.7724, + "kappa": 0.3482, + "mcc": 0.3649, + "roc_auc_weighted_ovr": 0.7886 + }, + "bytes": 1574683, + "sha256": "7f095da78e83abc96530b1d0f0f3552bd667002ee6a7fa5b887be9ce9028b353", + "reused": true + }, + { + "family": "dl", + "task": "cell_response", + "model": "FT_Transformer", + "protocol": "doi", + "selection": "weighted", + "file": "deep/doi/cell_response__ft_transformer.pt", + "source_fold": "doi_holdout", + "inner_score": 0.6168, + "params": { + "n_layers": 1, + "hidden_dim": 352, + "lr": 0.00011743525676125396, + "weight_decay": 0.005467630013091895, + "dropout": 0.3981692691675482, + "batch_size": 32, + "activation": "SELU", + "optimizer": "Adam", + "max_epochs": 154, + "class_weight": null + }, + "metrics": { + "accuracy": 0.7439, + "balanced_accuracy": 0.3921, + "macro_f1": 0.3739, + "weighted_f1": 0.7359, + "quadratic_kappa": 0.8202, + "kappa": 0.5504, + "mcc": 0.552, + "roc_auc_weighted_ovr": 0.9169 + }, + "bytes": 4426730, + "sha256": "1c54a866e50ffa88cda31b15c06a58a0b78332f20d655acc6b8f066a309cd0ad", + "reused": true + }, + { + "family": "dl", + "task": "cell_response", + "model": "MLP", + "protocol": "doi", + "selection": "weighted", + "file": "deep/doi/cell_response__mlp.pt", + "source_fold": "doi_holdout", + "inner_score": 0.6093, + "params": { + "n_layers": 5, + "hidden_dim": 448, + "lr": 0.0038377717552575347, + "weight_decay": 0.00034632526053519463, + "dropout": 0.11731103383390191, + "batch_size": 256, + "activation": "GELU", + "optimizer": "RMSprop", + "max_epochs": 216, + "class_weight": null + }, + "metrics": { + "accuracy": 0.5993, + "balanced_accuracy": 0.2724, + "macro_f1": 0.26, + "weighted_f1": 0.5585, + "quadratic_kappa": 0.2395, + "kappa": 0.218, + "mcc": 0.2235, + "roc_auc_weighted_ovr": 0.8217 + }, + "bytes": 3551373, + "sha256": "b3662c1fa368ff33f17d0cb6bbf016faa0b03a71a6c11af5fcdcd703284d6b18", + "reused": true + }, + { + "family": "dl", + "task": "cell_response", + "model": "NODE_Lite", + "protocol": "doi", + "selection": "weighted", + "file": "deep/doi/cell_response__node_lite.pt", + "source_fold": "doi_holdout", + "inner_score": 0.6241, + "params": { + "n_layers": 6, + "hidden_dim": 96, + "lr": 0.009842695582692119, + "weight_decay": 3.2006448599035196e-05, + "dropout": 0.1627390340258759, + "batch_size": 64, + "activation": "tanh", + "optimizer": "AdamW", + "max_epochs": 157, + "class_weight": "balanced" + }, + "metrics": { + "accuracy": 0.6098, + "balanced_accuracy": 0.3008, + "macro_f1": 0.3041, + "weighted_f1": 0.5942, + "quadratic_kappa": 0.4373, + "kappa": 0.284, + "mcc": 0.2888, + "roc_auc_weighted_ovr": 0.7939 + }, + "bytes": 494015, + "sha256": "f80f39fefd83008c8aece1f5e60dfcafdaef800a7d77771569922aac8a36df17", + "reused": true + }, + { + "family": "dl", + "task": "cell_response", + "model": "ResNet", + "protocol": "doi", + "selection": "weighted", + "file": "deep/doi/cell_response__resnet.pt", + "source_fold": "doi_holdout", + "inner_score": 0.5976, + "params": { + "n_layers": 1, + "hidden_dim": 448, + "lr": 0.00014096175149815865, + "weight_decay": 0.009133995846860976, + "dropout": 0.33167343078899725, + "batch_size": 128, + "activation": "SiLU", + "optimizer": "Adam", + "max_epochs": 197, + "class_weight": null + }, + "metrics": { + "accuracy": 0.6202, + "balanced_accuracy": 0.2858, + "macro_f1": 0.2763, + "weighted_f1": 0.5909, + "quadratic_kappa": 0.3867, + "kappa": 0.2834, + "mcc": 0.2868, + "roc_auc_weighted_ovr": 0.8074 + }, + "bytes": 1110183, + "sha256": "2b5bdcb36cd5555cf145afac8f8ea8a20a5ef478cb11b21191116eec41c7c907", + "reused": true + }, + { + "family": "dl", + "task": "cell_response", + "model": "TabNet_Lite", + "protocol": "doi", + "selection": "weighted", + "file": "deep/doi/cell_response__tabnet_lite.pt", + "source_fold": "doi_holdout", + "inner_score": 0.603, + "params": { + "n_layers": 3, + "hidden_dim": 480, + "lr": 0.006346577282966738, + "weight_decay": 0.0006593465821488544, + "dropout": 0.15630959598318916, + "batch_size": 128, + "activation": "GELU", + "optimizer": "Adam", + "max_epochs": 245, + "class_weight": null + }, + "metrics": { + "accuracy": 0.6167, + "balanced_accuracy": 0.3121, + "macro_f1": 0.2975, + "weighted_f1": 0.6163, + "quadratic_kappa": 0.5331, + "kappa": 0.3345, + "mcc": 0.3409, + "roc_auc_weighted_ovr": 0.8028 + }, + "bytes": 1828453, + "sha256": "6ee1895edc9370e00d1030c4722a13528362647fb1f54998a0202ffbc2c40fee", + "reused": true + }, + { + "family": "dl", + "task": "printability", + "model": "1D_CNN", + "protocol": "doi", + "selection": "weighted", + "file": "deep/doi/printability__1d_cnn.pt", + "source_fold": "doi_holdout", + "inner_score": 0.6418, + "params": { + "n_layers": 2, + "hidden_dim": 192, + "lr": 0.0004371440922464809, + "weight_decay": 0.0007604663958409753, + "dropout": 0.19075177963373807, + "batch_size": 256, + "activation": "ELU", + "optimizer": "RMSprop", + "max_epochs": 323, + "class_weight": null + }, + "metrics": { + "accuracy": 0.4181, + "balanced_accuracy": 0.2862, + "macro_f1": 0.2795, + "weighted_f1": 0.3997, + "quadratic_kappa": 0.0301, + "kappa": 0.0377, + "mcc": 0.0399, + "roc_auc_weighted_ovr": 0.5337 + }, + "bytes": 461649, + "sha256": "40276fcb8dedda1da9dc1d0df395d5ad0890dfede1afcf1b11881f0ee4ea4b0c", + "reused": true + }, + { + "family": "dl", + "task": "printability", + "model": "FT_Transformer", + "protocol": "doi", + "selection": "weighted", + "file": "deep/doi/printability__ft_transformer.pt", + "source_fold": "doi_holdout", + "inner_score": 0.6849, + "params": { + "n_layers": 1, + "hidden_dim": 448, + "lr": 0.00014096175149815865, + "weight_decay": 0.009133995846860976, + "dropout": 0.33167343078899725, + "batch_size": 128, + "activation": "SiLU", + "optimizer": "Adam", + "max_epochs": 197, + "class_weight": null + }, + "metrics": { + "accuracy": 0.5279, + "balanced_accuracy": 0.3434, + "macro_f1": 0.3386, + "weighted_f1": 0.5033, + "quadratic_kappa": 0.3773, + "kappa": 0.2106, + "mcc": 0.2137, + "roc_auc_weighted_ovr": 0.6997 + }, + "bytes": 7006547, + "sha256": "519bf6a0dd0666f4d9d80a7b9a02fb45ad87c6e02e0a9eecf2ab903b094b4075", + "reused": true + }, + { + "family": "dl", + "task": "printability", + "model": "MLP", + "protocol": "doi", + "selection": "weighted", + "file": "deep/doi/printability__mlp.pt", + "source_fold": "doi_holdout", + "inner_score": 0.6358, + "params": { + "n_layers": 4, + "hidden_dim": 320, + "lr": 0.0063193263783952215, + "weight_decay": 0.00012507787479502943, + "dropout": 0.34737533026378065, + "batch_size": 64, + "activation": "relu", + "optimizer": "Adam", + "max_epochs": 125, + "class_weight": null + }, + "metrics": { + "accuracy": 0.4443, + "balanced_accuracy": 0.3013, + "macro_f1": 0.2775, + "weighted_f1": 0.4236, + "quadratic_kappa": 0.1223, + "kappa": 0.0987, + "mcc": 0.1002, + "roc_auc_weighted_ovr": 0.5697 + }, + "bytes": 1465393, + "sha256": "6b8dc0b0db9c473caea70e344121fea265664fc1527a72e87226d33ddbc1561d", + "reused": true + }, + { + "family": "dl", + "task": "printability", + "model": "NODE_Lite", + "protocol": "doi", + "selection": "weighted", + "file": "deep/doi/printability__node_lite.pt", + "source_fold": "doi_holdout", + "inner_score": 0.6542, + "params": { + "n_layers": 6, + "hidden_dim": 512, + "lr": 0.007235342779315474, + "weight_decay": 2.7809192169581745e-05, + "dropout": 0.17325163380941638, + "batch_size": 32, + "activation": "SiLU", + "optimizer": "AdamW", + "max_epochs": 214, + "class_weight": "balanced" + }, + "metrics": { + "accuracy": 0.4338, + "balanced_accuracy": 0.3402, + "macro_f1": 0.3272, + "weighted_f1": 0.4351, + "quadratic_kappa": 0.3554, + "kappa": 0.1271, + "mcc": 0.1299, + "roc_auc_weighted_ovr": 0.5865 + }, + "bytes": 2609680, + "sha256": "187c094b621b4326cac4929a5bba7be896181fac88068451f0d9fc57b4746097", + "reused": true + }, + { + "family": "dl", + "task": "printability", + "model": "ResNet", + "protocol": "doi", + "selection": "weighted", + "file": "deep/doi/printability__resnet.pt", + "source_fold": "doi_holdout", + "inner_score": 0.6452, + "params": { + "n_layers": 2, + "hidden_dim": 128, + "lr": 0.007234279845665418, + "weight_decay": 0.002656813924114493, + "dropout": 0.2900211269531271, + "batch_size": 256, + "activation": "GELU", + "optimizer": "RMSprop", + "max_epochs": 253, + "class_weight": null + }, + "metrics": { + "accuracy": 0.5244, + "balanced_accuracy": 0.3739, + "macro_f1": 0.3777, + "weighted_f1": 0.5172, + "quadratic_kappa": 0.4254, + "kappa": 0.2311, + "mcc": 0.2315, + "roc_auc_weighted_ovr": 0.5963 + }, + "bytes": 227351, + "sha256": "5e314f0082ecb8d579c92c0a7a0968c9e5b9f7c1fc5ddb0482e6051628b541f8", + "reused": true + }, + { + "family": "dl", + "task": "printability", + "model": "TabNet_Lite", + "protocol": "doi", + "selection": "weighted", + "file": "deep/doi/printability__tabnet_lite.pt", + "source_fold": "doi_holdout", + "inner_score": 0.6531, + "params": { + "n_layers": 6, + "hidden_dim": 288, + "lr": 0.00017345566642360953, + "weight_decay": 0.0013795402040204172, + "dropout": 0.3282355145850693, + "batch_size": 64, + "activation": "ELU", + "optimizer": "AdamW", + "max_epochs": 327, + "class_weight": null + }, + "metrics": { + "accuracy": 0.4164, + "balanced_accuracy": 0.2299, + "macro_f1": 0.2115, + "weighted_f1": 0.3681, + "quadratic_kappa": -0.0354, + "kappa": -0.0455, + "mcc": -0.0482, + "roc_auc_weighted_ovr": 0.4257 + }, + "bytes": 2203428, + "sha256": "1b43f1e2c66f4a1743eae51e52b75fbb32274a2c0547409e7c838b7374a5957c", + "reused": true + }, + { + "family": "foundation", + "task": "printability", + "model": "TabPFN", + "protocol": "n/a", + "selection": "zero_shot", + "file": "foundation/printability__tabpfn.pkl", + "checkpoint": "tabpfn-v2.6-classifier-v2.6_default.ckpt", + "n_estimators": 8, + "context_rows": 2646, + "classes": [ + 0, + 1, + 2, + 3 + ], + "metrics": { + "accuracy": 0.7698, + "macro_f1": 0.7192, + "weighted_f1": 0.7696, + "mcc": 0.6443 + }, + "bytes": 21578, + "sha256": "f69c3b9604651aa3e4789a69d59a7544eeb31074e6277ea7a4763926aae86d4c", + "reused": true + }, + { + "family": "foundation", + "task": "printability", + "model": "TabPFN (thinking)", + "protocol": "n/a", + "selection": "zero_shot", + "file": "foundation/printability__tabpfn_thinking.pkl", + "checkpoint": "tabpfn-v2.6-classifier-v2.6_default.ckpt", + "n_estimators": 8, + "context_rows": 2646, + "classes": [ + 0, + 1, + 2, + 3 + ], + "metrics": { + "accuracy": 0.7792, + "macro_f1": 0.7383, + "weighted_f1": 0.7793, + "mcc": 0.6586 + }, + "bytes": 21590, + "sha256": "e7d5e330c3c552da1177c203502df36ad34823bf553a547e7a34b5328e1256ac", + "reused": true + }, + { + "family": "foundation", + "task": "printability", + "model": "TabICL", + "protocol": "n/a", + "selection": "zero_shot", + "file": "foundation/printability__tabicl.pkl", + "checkpoint": null, + "n_estimators": 8, + "context_rows": 2646, + "classes": [ + 0, + 1, + 2, + 3 + ], + "metrics": { + "accuracy": 0.8057, + "macro_f1": 0.7708, + "weighted_f1": 0.8065, + "mcc": 0.7024 + }, + "bytes": 21534, + "sha256": "f986088d0b6c6de01fe08ee2907db7ab123460bf5c25ac0348bbfbb772b8c3ca", + "reused": true + }, + { + "family": "foundation", + "task": "cell_response", + "model": "TabPFN", + "protocol": "n/a", + "selection": "zero_shot", + "file": "foundation/cell_response__tabpfn.pkl", + "checkpoint": "tabpfn-v2.6-classifier-v2.6_default.ckpt", + "n_estimators": 8, + "context_rows": 2646, + "classes": [ + 1, + 2, + 3, + 4, + 5 + ], + "metrics": { + "accuracy": 0.7811, + "macro_f1": 0.5345, + "weighted_f1": 0.7725, + "mcc": 0.6359 + }, + "bytes": 21466, + "sha256": "841200261e2e7838aa4026136523253e574dea2c98089460a8eb947705fbd04d", + "reused": true + }, + { + "family": "foundation", + "task": "cell_response", + "model": "TabPFN (thinking)", + "protocol": "n/a", + "selection": "zero_shot", + "file": "foundation/cell_response__tabpfn_thinking.pkl", + "checkpoint": "tabpfn-v2.6-classifier-v2.6_default.ckpt", + "n_estimators": 8, + "context_rows": 2646, + "classes": [ + 1, + 2, + 3, + 4, + 5 + ], + "metrics": { + "accuracy": 0.7736, + "macro_f1": 0.555, + "weighted_f1": 0.773, + "mcc": 0.6249 + }, + "bytes": 21479, + "sha256": "2ec859089ccbd723cb79eaa88807071b4db3bccf717bc98fdffde51288ae6210", + "reused": true + }, + { + "family": "foundation", + "task": "cell_response", + "model": "TabICL", + "protocol": "n/a", + "selection": "zero_shot", + "file": "foundation/cell_response__tabicl.pkl", + "checkpoint": null, + "n_estimators": 8, + "context_rows": 2646, + "classes": [ + 1, + 2, + 3, + 4, + 5 + ], + "metrics": { + "accuracy": 0.7698, + "macro_f1": 0.5456, + "weighted_f1": 0.7661, + "mcc": 0.6147 + }, + "bytes": 21429, + "sha256": "2baa4df60428ba939c88bb2395875331e17d38c905750940b2f7093b150538b9", + "reused": true + } + ] +} \ No newline at end of file diff --git a/deploy/models/foundation/cell_response__tabicl.pkl b/deploy/models/foundation/cell_response__tabicl.pkl new file mode 100644 index 0000000000000000000000000000000000000000..cbccd7e261c4248e168b38cac43b633394568740 --- /dev/null +++ b/deploy/models/foundation/cell_response__tabicl.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2baa4df60428ba939c88bb2395875331e17d38c905750940b2f7093b150538b9 +size 21429 diff --git a/deploy/models/foundation/cell_response__tabpfn.pkl b/deploy/models/foundation/cell_response__tabpfn.pkl new file mode 100644 index 0000000000000000000000000000000000000000..414be0ff53a99463867db39b6c9f78c34f9a2df3 --- /dev/null +++ b/deploy/models/foundation/cell_response__tabpfn.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:841200261e2e7838aa4026136523253e574dea2c98089460a8eb947705fbd04d +size 21466 diff --git a/deploy/models/foundation/cell_response__tabpfn_thinking.pkl b/deploy/models/foundation/cell_response__tabpfn_thinking.pkl new file mode 100644 index 0000000000000000000000000000000000000000..325f6a24a2a7c6c52b98e844dcd106447d1fd1dc --- /dev/null +++ b/deploy/models/foundation/cell_response__tabpfn_thinking.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ec859089ccbd723cb79eaa88807071b4db3bccf717bc98fdffde51288ae6210 +size 21479 diff --git a/deploy/models/foundation/printability__tabicl.pkl b/deploy/models/foundation/printability__tabicl.pkl new file mode 100644 index 0000000000000000000000000000000000000000..b0f92a94dc57bb08827d32745487c4eeb8b7c198 --- /dev/null +++ b/deploy/models/foundation/printability__tabicl.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f986088d0b6c6de01fe08ee2907db7ab123460bf5c25ac0348bbfbb772b8c3ca +size 21534 diff --git a/deploy/models/foundation/printability__tabpfn.pkl b/deploy/models/foundation/printability__tabpfn.pkl new file mode 100644 index 0000000000000000000000000000000000000000..3fef3e20ee8a24223262699caf3719017da3ddc7 --- /dev/null +++ b/deploy/models/foundation/printability__tabpfn.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f69c3b9604651aa3e4789a69d59a7544eeb31074e6277ea7a4763926aae86d4c +size 21578 diff --git a/deploy/models/foundation/printability__tabpfn_thinking.pkl b/deploy/models/foundation/printability__tabpfn_thinking.pkl new file mode 100644 index 0000000000000000000000000000000000000000..47bb46d3635bfe2a04fd06b4a4d0360860073ba5 --- /dev/null +++ b/deploy/models/foundation/printability__tabpfn_thinking.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e7d5e330c3c552da1177c203502df36ad34823bf553a547e7a34b5328e1256ac +size 21590 diff --git a/deploy/models/preprocessors/feature_names.pkl b/deploy/models/preprocessors/feature_names.pkl new file mode 100644 index 0000000000000000000000000000000000000000..cd93bcc7b4d3afe6d9cc0e144eca6ca121d53628 --- /dev/null +++ b/deploy/models/preprocessors/feature_names.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de846baea9ea14454cadf5e04a965975074c4cb8413a8a745e8fcbd33a4b7f3c +size 1531 diff --git a/deploy/models/preprocessors/input_columns.pkl b/deploy/models/preprocessors/input_columns.pkl new file mode 100644 index 0000000000000000000000000000000000000000..3790432dac7b109a5962ecedc26676a3deb6ad29 --- /dev/null +++ b/deploy/models/preprocessors/input_columns.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c9505e1cc75563f88d19f125511f19c113180c4174551192ecbfdce5391dc117 +size 1454 diff --git a/deploy/models/preprocessors/manifest.json b/deploy/models/preprocessors/manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..e7611414d302291257cb1198c6e361f1062898ee --- /dev/null +++ b/deploy/models/preprocessors/manifest.json @@ -0,0 +1,38 @@ +{ + "created_utc": "2026-09-02T17:42:00+00:00", + "python": "3.14.3", + "sklearn": "1.8.0", + "numpy": "2.4.4", + "pandas": "2.3.3", + "config": { + "imputer_estimator": "extra_trees", + "imputer_max_iter": 10, + "ambient_strategy": "constant", + "ambient_temperature_c": 22.0, + "random_state": 42 + }, + "dataset": { + "rows": 2646, + "input_columns": 139, + "output_features": 153, + "indicator_features": 7 + }, + "files": { + "feature_names.pkl": { + "bytes": 1531, + "sha256": "de846baea9ea14454cadf5e04a965975074c4cb8413a8a745e8fcbd33a4b7f3c" + }, + "input_columns.pkl": { + "bytes": 1454, + "sha256": "c9505e1cc75563f88d19f125511f19c113180c4174551192ecbfdce5391dc117" + }, + "preprocessor.pkl": { + "bytes": 10079, + "sha256": "bc836520d819ab28bc4720dbb76130086b4da9932f5204bd41fd6efb213812a5" + }, + "target_schema.json": { + "bytes": 494, + "sha256": "63f239029b5935e444800fd38d4c90c93153ec937cd7d0d2726c56c734fd83c0" + } + } +} \ No newline at end of file diff --git a/deploy/models/preprocessors/preprocessor.pkl b/deploy/models/preprocessors/preprocessor.pkl new file mode 100644 index 0000000000000000000000000000000000000000..c8fa741f2ecf284192da8a020114f4b1fb917805 --- /dev/null +++ b/deploy/models/preprocessors/preprocessor.pkl @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc836520d819ab28bc4720dbb76130086b4da9932f5204bd41fd6efb213812a5 +size 10079 diff --git a/deploy/models/preprocessors/target_schema.json b/deploy/models/preprocessors/target_schema.json new file mode 100644 index 0000000000000000000000000000000000000000..2f394642dff56ae6fe0dbf1c2c52883edc23d316 --- /dev/null +++ b/deploy/models/preprocessors/target_schema.json @@ -0,0 +1,16 @@ +{ + "Printability": { + "0": "the ink was not extruded", + "1": "the ink behaved like a liquid, formed beads", + "2": "extrudable but not optimised", + "3": "extrudable and optimised" + }, + "Cell Response": { + "1": "no cells included - not applicable", + "2": "inadequate short-term response", + "3": "good short-term response", + "4": "good short-term, poor long-term", + "5": "good short- and long-term response" + }, + "acellular_token": "NoCellCultured" +} \ No newline at end of file diff --git a/mlate/__init__.py b/mlate/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/mlate/artifacts.py b/mlate/artifacts.py new file mode 100644 index 0000000000000000000000000000000000000000..a1a6520e8007ff2d22941288be0a0b4038c779f9 --- /dev/null +++ b/mlate/artifacts.py @@ -0,0 +1,204 @@ +""" +Fitted-artefact persistence +=========================== + +Fitting a preprocessor is expensive, so it is done once and reused. Two kinds +of artefact, with different purposes: + +fold preprocessors + Fitted on one fold's TRAINING rows only, cached under results/. Every model + scored on that fold reuses the same object, so the imputer is fitted once + per fold rather than once per model per hyper-parameter trial. Cached by a + hash of the training row set, so tasks that share a partition - printability + and cell_response run on the same 2,646 rows - share one artefact. + These are working files and are NOT committed. + +deployment preprocessor + Fitted on the whole dataset and shipped with the release. This is what the + web application loads to transform a user's formulation, and what a reader + needs to reproduce inference. Committed, alongside the feature-name list and + a manifest recording library versions and file hashes. + +A note on size: IterativeImputer keeps every fitted sub-model so it can replay +the imputation sequence at transform time, so an artefact grows linearly with +max_iter (roughly 8 MB per iteration here). Keep cfg.IMPUTER_MAX_ITER at a value +justified by preprocessing/sweep_iterations.py, or the release becomes +undistributable. +""" + +from __future__ import annotations + +import hashlib +import json +import platform +from datetime import datetime, timezone +from pathlib import Path + +import joblib +import numpy as np +import pandas as pd +import sklearn + +from mlate import config as cfg +from mlate.dataset import Columns +from mlate.pipeline import build_preprocessor, feature_names + +CACHE_DIR = cfg.step_dir("02_preprocessing", "models") / "fold_cache" +RELEASE_DIR = cfg.PREPROCESSOR_DIR +COMPRESS = 3 # joblib zlib level; trees compress well + + +def _config_fingerprint() -> str: + """Anything that changes the fitted object must change the cache key.""" + payload = { + "estimator": cfg.IMPUTER_ESTIMATOR, + "max_iter": cfg.IMPUTER_MAX_ITER, + "ambient_strategy": cfg.AMBIENT_STRATEGY, + "ambient_value": cfg.AMBIENT_TEMPERATURE_C, + "seed": cfg.RANDOM_STATE, + "sklearn": sklearn.__version__, + } + return hashlib.sha256( + json.dumps(payload, sort_keys=True).encode()).hexdigest()[:12] + + +def cache_key(train_idx: np.ndarray, columns: Columns | None = None) -> str: + """ + Cache identity of a fitted fold preprocessor. + + The predictor list is part of the key, not just the rows and the imputation + settings. A preprocessor is fitted against a specific column contract, so + two runs that disagree about which columns are predictors must not share an + artefact - otherwise changing the input set silently reloads a transformer + fitted on the old one, and the mismatch surfaces much later as a shape + error, or does not surface at all. + """ + rows = hashlib.sha256( + np.asarray(sorted(train_idx), dtype=np.int64).tobytes()).hexdigest()[:16] + if columns is None: + return f"{rows}_{_config_fingerprint()}" + cols = hashlib.sha256( + json.dumps(list(columns.predictors)).encode()).hexdigest()[:8] + return f"{rows}_{_config_fingerprint()}_{cols}" + + +def fold_preprocessor(df: pd.DataFrame, columns: Columns, + train_idx: np.ndarray, use_cache: bool = True): + """ + Fitted preprocessor for one fold's training rows. + + Returns (preprocessor, path, was_cached). Fitting uses training rows only, + so the cached object can be reused for every model scored on that fold + without leaking anything. + """ + CACHE_DIR.mkdir(parents=True, exist_ok=True) + path = CACHE_DIR / f"pre_{cache_key(train_idx, columns)}.pkl" + if use_cache and path.exists(): + return joblib.load(path), path, True + + pre = build_preprocessor(columns) + pre.fit(df[columns.predictors].iloc[train_idx]) + joblib.dump(pre, path, compress=COMPRESS) + return pre, path, False + + +def build_release(df: pd.DataFrame, columns: Columns) -> dict: + """ + Fit and save the artefacts that ship with the paper. + + The deployment preprocessor is fitted on every row on purpose: at inference + time there is no held-out set to protect, and a user's formulation should be + transformed using everything the dataset knows. It is never used to score a + model - that is what the fold preprocessors are for. + """ + RELEASE_DIR.mkdir(parents=True, exist_ok=True) + + # Exactly the same object the models were trained behind, fitted on every + # row. There was once a separate lightweight deployment variant, needed + # only because an IterativeImputer-based pipeline weighed hundreds of + # megabytes; with median imputation the real pipeline is ~10 kB, so keeping + # a second implementation in step would only invite the two to drift apart. + pre = build_preprocessor(columns).fit(df[columns.predictors]) + names = feature_names(pre) + + joblib.dump(pre, RELEASE_DIR / "preprocessor.pkl", compress=COMPRESS) + joblib.dump(list(columns.predictors), RELEASE_DIR / "input_columns.pkl", + compress=COMPRESS) + joblib.dump(names, RELEASE_DIR / "feature_names.pkl", compress=COMPRESS) + + # Targets are already integer-coded; ship the human-readable meanings so the + # application never has to hard-code them. + schema = { + "Printability": { + "0": "the ink was not extruded", + "1": "the ink behaved like a liquid, formed beads", + "2": "extrudable but not optimised", + "3": "extrudable and optimised", + }, + "Cell Response": { + "1": "no cells included - not applicable", + "2": "inadequate short-term response", + "3": "good short-term response", + "4": "good short-term, poor long-term", + "5": "good short- and long-term response", + }, + "acellular_token": cfg.ACELLULAR_TOKEN, + } + (RELEASE_DIR / "target_schema.json").write_text( + json.dumps(schema, indent=2), encoding="utf-8") + + manifest = { + "created_utc": datetime.now(timezone.utc).isoformat(timespec="seconds"), + "python": platform.python_version(), + "sklearn": sklearn.__version__, + "numpy": np.__version__, + "pandas": pd.__version__, + "config": { + "imputer_estimator": cfg.IMPUTER_ESTIMATOR, + "imputer_max_iter": cfg.IMPUTER_MAX_ITER, + "ambient_strategy": cfg.AMBIENT_STRATEGY, + "ambient_temperature_c": cfg.AMBIENT_TEMPERATURE_C, + "random_state": cfg.RANDOM_STATE, + }, + "dataset": { + "rows": int(len(df)), + "input_columns": len(columns.predictors), + "output_features": len(names), + "indicator_features": sum("[reported]" in n for n in names), + }, + "files": {}, + } + for p in sorted(RELEASE_DIR.iterdir()): + if p.name == "manifest.json": + continue + manifest["files"][p.name] = { + "bytes": p.stat().st_size, + "sha256": hashlib.sha256(p.read_bytes()).hexdigest(), + } + (RELEASE_DIR / "manifest.json").write_text( + json.dumps(manifest, indent=2), encoding="utf-8") + return manifest + + +def load_release(): + """Load the shipped preprocessor. Used by the web application.""" + pre = joblib.load(RELEASE_DIR / "preprocessor.pkl") + cols = joblib.load(RELEASE_DIR / "input_columns.pkl") + names = joblib.load(RELEASE_DIR / "feature_names.pkl") + schema = json.loads((RELEASE_DIR / "target_schema.json").read_text("utf-8")) + return pre, cols, names, schema + + +if __name__ == "__main__": + from mlate.dataset import load_dataset + + frame, cols = load_dataset() + info = build_release(frame, cols) + total = sum(f["bytes"] for f in info["files"].values()) + print(f"release artefacts -> {RELEASE_DIR}") + for name, meta in info["files"].items(): + print(f" {name:<28} {meta['bytes'] / 2**20:8.1f} MB") + print(f" {'TOTAL':<28} {total / 2**20:8.1f} MB" + + (" << exceeds GitHub's 100 MB file limit" + if any(f["bytes"] > 100 * 2**20 for f in info["files"].values()) + else "")) diff --git a/mlate/clustering.py b/mlate/clustering.py new file mode 100644 index 0000000000000000000000000000000000000000..14e5a56ded34aa6f43d403e4ccded4474153baf6 --- /dev/null +++ b/mlate/clustering.py @@ -0,0 +1,198 @@ +""" +Clustering helpers shared by step 03 +==================================== + +The feature matrix is 153 columns, but 130 of them are biomaterial +concentrations that are zero in roughly 95% of rows, so Euclidean distance +risks being dominated by the sparse block: two formulations judged similar +mostly because they both omit the same 120 materials. + +An earlier version of this module applied PCA first and justified it on exactly +that ground. The justification did not survive being tested. Measuring the +Spearman correlation between pairwise distance and the number of jointly-zero +columns gives -0.775 in the raw space and -0.770 after PCA - the projection +does not decouple distance from sparsity, because the leading components are +themselves built out of the sparsity pattern. `reduce()` is therefore offered +but is no longer the default, and the raw space, which is also what the +submitted manuscript used, is primary. 03_clustering/optimal_k.py recomputes +that diagnostic on every run. + +Nothing here selects a clustering. It provides the algorithms, the internal +indices and a resampling-based stability score; the choice is made in +03_clustering/run.py from the table these produce. +""" + +from __future__ import annotations + +import numpy as np +import pandas as pd +from sklearn.cluster import (AgglomerativeClustering, BisectingKMeans, + KMeans, MiniBatchKMeans) +from sklearn.decomposition import PCA +from sklearn.metrics import (adjusted_mutual_info_score, adjusted_rand_score, + calinski_harabasz_score, davies_bouldin_score, + silhouette_score) +from sklearn.mixture import GaussianMixture + +from mlate import config as cfg + +PCA_VARIANCE = 0.95 +K_RANGE = range(2, 31) +N_BOOTSTRAP = 20 +BOOTSTRAP_FRACTION = 0.80 + + +def reduce(matrix: np.ndarray, variance: float = PCA_VARIANCE, + seed: int = cfg.RANDOM_STATE) -> tuple[np.ndarray, PCA]: + """PCA to a fixed explained-variance target.""" + pca = PCA(n_components=variance, svd_solver="full", random_state=seed) + return pca.fit_transform(matrix), pca + + +def algorithms(seed: int = cfg.RANDOM_STATE, space: str = "raw") -> dict: + """ + k-parameterised algorithms. HDBSCAN chooses its own k and is separate. + + The four of the submitted manuscript - KMeans, MiniBatchKMeans, + BisectingKMeans and Agglomerative/Ward - plus GaussianMixture. MiniBatch + appears in the published Table S3, so omitting it would leave a published + row with no counterpart in the revision. + + GaussianMixture takes full covariance only in a reduced space. In the raw + 153-column matrix, where 130 columns are zero in ~95% of rows, a full + per-component covariance is rank-deficient: it either fails or is rescued + by regularisation into a number that means nothing. Diagonal covariance is + the honest choice there, and the restriction is a property of the space + rather than a tuning preference. + """ + cov = "full" if space == "pca" else "diag" + return { + "KMeans": lambda k: KMeans(n_clusters=k, n_init=20, random_state=seed), + "MiniBatchKMeans": lambda k: MiniBatchKMeans( + n_clusters=k, n_init=10, random_state=seed, batch_size=1024), + "BisectingKMeans": lambda k: BisectingKMeans( + n_clusters=k, n_init=10, random_state=seed), + "AgglomerativeWard": lambda k: AgglomerativeClustering( + n_clusters=k, linkage="ward"), + "GaussianMixture": lambda k: GaussianMixture( + n_components=k, covariance_type=cov, n_init=5, random_state=seed, + reg_covar=1e-5), + } + + +def fit_predict(model, X: np.ndarray) -> np.ndarray: + """GaussianMixture splits fit and predict; the cluster estimators do not.""" + if hasattr(model, "fit_predict"): + return np.asarray(model.fit_predict(X)) + return np.asarray(model.fit(X).predict(X)) + + +def indices(X: np.ndarray, labels: np.ndarray) -> dict: + """Internal validity indices, or NaN where a single cluster makes them undefined.""" + uniq = np.unique(labels[labels >= 0]) + if len(uniq) < 2: + return {"silhouette": np.nan, "davies_bouldin": np.nan, + "calinski_harabasz": np.nan} + keep = labels >= 0 # HDBSCAN marks noise as -1 + return { + "silhouette": float(silhouette_score(X[keep], labels[keep])), + "davies_bouldin": float(davies_bouldin_score(X[keep], labels[keep])), + "calinski_harabasz": float(calinski_harabasz_score(X[keep], labels[keep])), + } + + +def _assign_by_centroid(X: np.ndarray, centres: np.ndarray) -> np.ndarray: + d = ((X[:, None, :] - centres[None, :, :]) ** 2).sum(axis=2) + return d.argmin(axis=1) + + +def stability(X: np.ndarray, make, k: int, n_boot: int = N_BOOTSTRAP, + fraction: float = BOOTSTRAP_FRACTION, + seed: int = cfg.RANDOM_STATE) -> float: + """ + Mean adjusted Rand index between the full-data labelling and labellings + refitted on resampled subsets. + + A high silhouette on one fit says the partition is compact; it does not say + the partition would reappear if the corpus had been assembled slightly + differently. For a literature-mined dataset that second question is the + important one, so it gets its own number. Subset labels are carried back to + all rows by nearest centroid, which lets the linkage methods - which have no + predict() - be scored the same way as the rest. + """ + rng = np.random.default_rng(seed) + reference = fit_predict(make(k), X) + scores = [] + for _ in range(n_boot): + idx = rng.choice(len(X), size=int(fraction * len(X)), replace=False) + sub = fit_predict(make(k), X[idx]) + centres = np.vstack([X[idx][sub == c].mean(axis=0) + for c in np.unique(sub)]) + scores.append(adjusted_rand_score(reference, + _assign_by_centroid(X, centres))) + return float(np.mean(scores)) + + + +# The plane every clustering figure is drawn in +# -------------------------------------------- +REFERENCE_ALGORITHM = "BisectingKMeans" +REFERENCE_K = 4 + + +def reference_plane(X: np.ndarray, seed: int = cfg.RANDOM_STATE): + """ + One two-dimensional view, shared by Figures 4, 5, S15 and S17. + + Every clustering figure must place a given formulation at the same point, + or a reader comparing two figures is comparing coordinate systems rather + than partitions. Fitting each figure's own discriminants - as an earlier + version did - satisfied nobody: it showed each partition at its best but + made the figures mutually incomparable. + + The plane chosen is the discriminants of the secondary partition, bisecting + k-means at k = 4, and the choice is empirical rather than aesthetic. It is + the only candidate that keeps BOTH reported partitions clearly separated: + + in the k=3 plane in the k=4 plane + k-means k = 3 0.703 0.589 + bisecting k = 4 0.529 0.745 + + The k = 3 plane costs the four-cluster partition 0.216 of silhouette; the + k = 4 plane costs the three-cluster partition 0.114 and still separates it + plainly. Neither UMAP nor the leading principal components is usable here - + UMAP scores -0.063 on the reported partition, and two principal components + of a 153-column matrix collapse the corpus into an overplotted band. + + Because the plane is fitted to labels it flatters the partition it came + from, so no partition is judged by its appearance in it. Separability is + always reported from each partition's own discriminants instead. + """ + from sklearn.cluster import BisectingKMeans + from sklearn.discriminant_analysis import LinearDiscriminantAnalysis + + labels = BisectingKMeans(n_clusters=REFERENCE_K, n_init=10, + random_state=seed).fit_predict(X) + return (LinearDiscriminantAnalysis(n_components=2) + .fit(X, labels).transform(X)) + + +def agreement(labels: np.ndarray, other: pd.Series) -> dict: + """ + How much of a partition is explained by an external label. + + ARI is reported alongside AMI because DOI has 222 categories against a + handful of clusters, a regime where ARI is pulled toward zero regardless of + the real association. Purity is the plainest of the three: the share of rows + sitting in the most common external category of their own cluster. + """ + other = pd.Series(other).astype(str).to_numpy() + keep = labels >= 0 + lab, oth = labels[keep], other[keep] + purity = (pd.DataFrame({"c": lab, "o": oth}) + .groupby("c")["o"].agg(lambda s: s.value_counts().iloc[0]).sum()) + return { + "ARI": float(adjusted_rand_score(oth, lab)), + "AMI": float(adjusted_mutual_info_score(oth, lab)), + "purity": float(purity / len(lab)), + } diff --git a/mlate/config.py b/mlate/config.py new file mode 100644 index 0000000000000000000000000000000000000000..f3dcc50f7a05a92e898fbc5a8e1a711686a992b1 --- /dev/null +++ b/mlate/config.py @@ -0,0 +1,165 @@ +""" +MLATE V3 β€” paths and global constants +===================================== + +Every stage imports from here. Nothing else hard-codes a path. +""" + +from __future__ import annotations + +import os +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent + +# ───────────────────────────────────────────────────────────────────────────── +# PATHS +# ───────────────────────────────────────────────────────────────────────────── +DATA_DIR = ROOT / "01_data" +RAW_DIR = DATA_DIR / "raw" +PROCESSED_DIR = DATA_DIR / "processed" +REFERENCE_DIR = DATA_DIR / "reference" + +RESULTS_DIR = ROOT / "results" + +# Pipeline steps. Each has one results folder holding its figures, tables and +# models, so an output is always next to the step that produced it. +STEPS = ("01_data", "02_preprocessing", "03_clustering", + "04_machine_learning", "05_deep_learning", "06_webapp", + "07_wssq") + + +def step_dir(step: str, kind: str | None = None) -> Path: + """results///, created on demand. kind: figures|tables|models.""" + if step not in STEPS: + raise ValueError(f"unknown step {step!r}; expected one of {STEPS}") + path = RESULTS_DIR / step / kind if kind else RESULTS_DIR / step + path.mkdir(parents=True, exist_ok=True) + return path + +DATASET = PROCESSED_DIR / "MLATE_V3_dataset.xlsx" +DATASET_CSV = PROCESSED_DIR / "MLATE_V3_dataset.csv" +INTERNAL = PROCESSED_DIR / "MLATE_V3_internal.xlsx" +AUDIT = PROCESSED_DIR / "MLATE_V3_audit.xlsx" +DICTIONARY = PROCESSED_DIR / "MLATE_V3_dictionary.xlsx" + +DEPLOY_DIR = ROOT / "deploy" # uploaded to Hugging Face, not git +MODEL_DIR = DEPLOY_DIR / "models" +PREPROCESSOR_DIR = MODEL_DIR / "preprocessors" + +TAXONOMY = REFERENCE_DIR / "biomaterial_taxonomy.csv" +CELLLINE_SYNONYMS = REFERENCE_DIR / "cellline_synonyms.csv" + +# The immutable source workbook. Read-only, never written by this repo, and +# not redistributed here - data/processed/ holds the published version. +RAW_WORKBOOK = Path( + "G:/My Drive/Papers/MLATE V3_Revision/MLATE_V3_code_and_dataset_2026-08-31" + "/MLATE_V3_code_and_dataset/data/raw/MLATE_V3_dataset.xlsx" +) +RAW_SHEET = "merged_dataset" + +# ───────────────────────────────────────────────────────────────────────────── +# COLUMN CONTRACT +# ───────────────────────────────────────────────────────────────────────────── +META_FRONT = ["Reference", "DOI", "target_tissue", "target_tissue_all", + "is_cancer_model"] +META_BACK = ["dup_group_id", "label_conflict"] +CELL_COLS = ["Cell Line", "Cell Density (million cells/mL)"] +TARGETS = ["Printability", "Cell Response"] + +PRINT_PARAMS = [ + "Physical Crosslinking Duration (s)", + "Photo Crosslinking Duration (s)", + "Extrusion Pressure (kPa)", + "Nozzle Movement Speed (mm/s)", + "Nozzle Diameter (\u00b5m)", + "Syringe Temperature (\u00b0C)", + "Substrate Temperature (\u00b0C)", +] + +# Printed at ambient unless the study says otherwise, so a missing value is +# informative rather than unknown. See preprocessing/imputation.py. +AMBIENT_COLUMNS = [ + "Syringe Temperature (\u00b0C)", + "Substrate Temperature (\u00b0C)", +] +# 22 C is both the modal and the median reported value for these two columns, +# ahead of 25 C, so it is what these authors actually mean by 'room +# temperature'. The masking test in preprocessing/validate_imputation.py +# confirms 22 beats 25 on both columns (Substrate MAE 6.72 vs 8.52, Syringe +# 11.03 vs 11.51). +AMBIENT_TEMPERATURE_C = 22.0 + +# How the ambient columns are filled. +# "model_based" iterative imputation + [reported] flag. Default. The masking +# test puts it well ahead of any constant (Substrate MAE 2.17 +# vs 6.72, Syringe 7.07 vs 11.03). +# "constant" AMBIENT_TEMPERATURE_C + [reported] flag - the pure domain +# assumption, kept because the masking test is scored only on +# reported values, which are the deliberately heated and +# cooled runs, and so is biased against a constant. +# Either way the [reported] indicator is retained, which is what actually lets +# a model distinguish a measured 22 C from an assumed one. +AMBIENT_STRATEGY = "constant" + +# Printing parameters with no physical default; a blank genuinely means unknown +# and they go to the model-based imputer. +UNKNOWN_IF_MISSING = [c for c in PRINT_PARAMS if c not in AMBIENT_COLUMNS] + +ACELLULAR_TOKEN = "NoCellCultured" + +# target_tissue values that name no organ; collapsed for leave-one-tissue-out. +NON_ORGAN_TISSUES = { + "undifferentiated_stem_cell", "acellular", + "general_biocompatibility", "non_mammalian", +} + +# ───────────────────────────────────────────────────────────────────────────── +# EXPERIMENT SETTINGS +# ───────────────────────────────────────────────────────────────────────────── +RANDOM_STATE = 42 + +# Outer cross-validation folds for the random and DOI-grouped protocols. +# Ten, matching the 10-fold scheme the submitted pipeline used inside its +# hyper-parameter search, so the revision's outer evaluation is at least as +# fine-grained as the tuning it reports. Checked against both protocols: every +# fold retains all classes on both targets, and grouped test folds hold 257-288 +# rows for the 2,646-row tasks. +N_FOLDS = 10 + +# ── compute budget ─────────────────────────────────────────────────────────── +# 80% of the logical cores, leaving headroom so the machine stays usable. +# Passed as n_jobs to every estimator that accepts it. +CPU_TOTAL = os.cpu_count() or 8 +CPU_FRACTION = 0.80 +RAM_FRACTION = 0.85 +GPU_MEMORY_FRACTION = 0.90 +N_JOBS = max(1, int(CPU_FRACTION * CPU_TOTAL)) + +# IterativeImputer settings, chosen from preprocessing/sweep_iterations.py. +# That sweep covered max_iter 1-50 x 16/32/64 trees: nMAE spans only 0.534-0.591 +# across all 18 configurations and moves non-monotonically with max_iter, which +# is the signature of noise rather than improvement. The one real effect is that +# max_iter=1 is consistently worst (0.585 mean vs 0.561 for >=2). 10 iterations +# therefore sits comfortably past the point of diminishing returns while costing +# ~8 s and ~10 MB per fit; 100 would cost 10x for no measurable gain. +IMPUTER_MAX_ITER = 10 +IMPUTER_N_ESTIMATORS = 32 + +# Regressor inside IterativeImputer: "extra_trees" or "xgboost". +# Both are benchmarked in preprocessing/validate_imputation.py; the +# manuscript's Methods must name whichever is set here. +# How the printing parameters are filled: "median" (default) or +# "model_based". Study-grouped masking shows the median wins outright +# (nMAE 1.30 vs 1.54 / 1.90 / 1.99 for ExtraTrees / XGBoost / KNN) and +# that every method has a negative R^2 - these parameters are not +# predictable across studies. See preprocessing/validate_imputation.py. +PRINTING_IMPUTATION = "median" +IMPUTER_ESTIMATOR = "extra_trees" # used only when model_based +TEST_SIZE = 0.20 # random hold-out, for the interpolation estimate +MIN_TISSUE_SAMPLES = 40 # tissues smaller than this are pooled in LOTO + +# Cell Response class 1 means "no cells were included", not a biological +# outcome, so the cellular-only task drops it (see reviewer comment R2-3). +CELL_RESPONSE_CELLULAR = (2, 3, 4, 5) +PRINTABILITY_CLASSES = (0, 1, 2, 3) diff --git a/mlate/dataset.py b/mlate/dataset.py new file mode 100644 index 0000000000000000000000000000000000000000..575da256e758d04157cfadf875bb66606eed389d --- /dev/null +++ b/mlate/dataset.py @@ -0,0 +1,116 @@ +""" +Dataset access and the column contract +====================================== + +One loader, one definition of which columns are what. Every stage uses these +so a column can never be treated as a predictor in one place and metadata in +another. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +import pandas as pd + +from mlate import config as cfg + + +@dataclass(frozen=True) +class Columns: + """Which columns play which role, resolved against an actual frame.""" + biomaterials: list[str] + print_params: list[str] + cell_line: str + cell_density: str + targets: list[str] + + @property + def numeric_predictors(self) -> list[str]: + return [*self.biomaterials, self.cell_density, *self.print_params] + + @property + def predictors(self) -> list[str]: + return [*self.biomaterials, self.cell_line, self.cell_density, + *self.print_params] + + +def column_groups(df: pd.DataFrame) -> Columns: + """ + Resolve the column contract against a loaded frame. + + Biomaterials are defined positionally - everything between the metadata + block and 'Cell Line' - because their names change as the dataset grows. + """ + cols = list(df.columns) + start = len(cfg.META_FRONT) + biomaterials = cols[start:cols.index(cfg.CELL_COLS[0])] + + missing = [c for c in cfg.PRINT_PARAMS + cfg.TARGETS + cfg.CELL_COLS + if c not in cols] + if missing: + raise ValueError(f"dataset is missing expected columns: {missing}") + overlap = set(biomaterials) & set(cfg.PRINT_PARAMS + cfg.TARGETS) + if overlap: + raise ValueError(f"biomaterial block overlaps other roles: {overlap}") + + return Columns( + biomaterials=biomaterials, + print_params=list(cfg.PRINT_PARAMS), + cell_line=cfg.CELL_COLS[0], + cell_density=cfg.CELL_COLS[1], + targets=list(cfg.TARGETS), + ) + + +def load_dataset(path=None) -> tuple[pd.DataFrame, Columns]: + """Load the published dataset and its column contract.""" + df = pd.read_excel(path or cfg.DATASET) + return df, column_groups(df) + + +def load_taxonomy() -> pd.DataFrame: + return pd.read_csv(cfg.TAXONOMY) + + +def modeling_tissue(df: pd.DataFrame) -> pd.Series: + """ + Tissue label used for leave-one-tissue-out. + + Collapses the four non-anatomical target_tissue values into + 'not_organ_specific', then pools any tissue with too few samples to give a + meaningful held-out fold. + """ + t = df["target_tissue"].where( + ~df["target_tissue"].isin(cfg.NON_ORGAN_TISSUES), "not_organ_specific") + small = t.value_counts().loc[lambda s: s < cfg.MIN_TISSUE_SAMPLES].index + return t.where(~t.isin(small), "other_small_tissue") + + +def is_cellular(df: pd.DataFrame) -> pd.Series: + """True where the construct actually contained cells.""" + return df[cfg.CELL_COLS[0]] != cfg.ACELLULAR_TOKEN + + +def target_frame(df: pd.DataFrame, task: str) -> tuple[pd.DataFrame, pd.Series]: + """ + Rows and labels for one prediction task. + + 'printability' all rows, classes 0-3 + 'cell_response' all rows, classes 1-5 (comparable to the submitted + version, where class 1 means 'no cells') + 'cell_response_cellular' cellular rows only, classes 2-5 (reviewer R2-3) + """ + if task == "printability": + return df, df["Printability"].astype(int) + if task == "cell_response": + return df, df["Cell Response"].astype(int) + if task == "cell_response_cellular": + mask = is_cellular(df) & df["Cell Response"].isin( + cfg.CELL_RESPONSE_CELLULAR) + sub = df[mask] + return sub, sub["Cell Response"].astype(int) + raise ValueError(f"unknown task: {task}") + + +TASKS = ("printability", "cell_response", "cell_response_cellular") diff --git a/mlate/deep.py b/mlate/deep.py new file mode 100644 index 0000000000000000000000000000000000000000..9ca379f76601264f6c5b2344d8ee0e0e1ea9945b --- /dev/null +++ b/mlate/deep.py @@ -0,0 +1,410 @@ +""" +Tabular deep-learning architectures +=================================== + +The six architectures the submitted manuscript benchmarked, ported from +`dl_benchmark_revised_v2.py` so the revision evaluates the same models, and +restructured so they run on the same folds, the same preprocessor and the same +metric panel as the conventional models in step 04. + +Why that restructuring matters +------------------------------ +The submitted version evaluated conventional ML under an 80:20 split and the +neural models under a 70:15:15 split, and says so in its own Discussion: the +two groups could only be compared "as benchmark-level comparisons rather than +strictly identical test-split comparisons". Putting every model on one set of +folds removes that caveat, and makes the ML-vs-DL comparison in the revision a +real one. + +What changed from the submitted implementation, and why +------------------------------------------------------- +* **Feature tokenisation is vectorised.** FT-Transformer built one + `nn.Linear(1, d_token)` per input feature - 153 separate modules, each + launching its own kernel. The replacement is a single (n_features, d_token) + weight and bias applied with a broadcast multiply, which is mathematically + identical (per-feature affine map) and far faster. +* **Early stopping replaces a fixed epoch count.** The submitted objective + searched `num_epochs` in [200, 600] and always ran to the end. Validation + loss on this dataset plateaus long before that - the manuscript's own + Figure S10 shows several architectures diverging after the first phase - so + training past the plateau bought overfitting, not accuracy. +* **Class weighting is available to the search.** Both targets are imbalanced + (Cell Response class 1 is 60.3% of rows). An unweighted cross-entropy lets a + network reach a good loss by ignoring minority classes, which is the failure + the macro-F1 metric is there to catch. +* **ONNX export, torchviz and the Tee logger are gone.** None of them affect a + result, and each pulled in a dependency the released environment would have + to carry. + +Determinism +----------- +Seeded per unit. Full CUDA determinism is not forced: it would disable the +cuDNN autotuner and roughly halve throughput, and these results are reported as +cross-validated means, not as single reproducible fits. The seed is recorded so +a run is repeatable to within GPU non-determinism. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Callable + +import numpy as np + +from mlate import config as cfg + +SEED = cfg.RANDOM_STATE + + +# ── activations and optimisers, as the submitted pipeline defined them ─────── + +def _torch(): + import torch + return torch + + +def activations() -> dict: + import torch.nn as nn + return {"relu": nn.ReLU, "tanh": nn.Tanh, "GELU": nn.GELU, + "SELU": nn.SELU, "ELU": nn.ELU, "SiLU": nn.SiLU} + + +def optimisers() -> dict: + import torch.optim as optim + return {"Adam": optim.Adam, "AdamW": optim.AdamW, + "RMSprop": optim.RMSprop, "SGD": optim.SGD} + + +# ── architectures ──────────────────────────────────────────────────────────── + +def _build_modules(): + """ + Defined inside a function so importing this module does not import torch. + + Step 04 imports `mlate` in fifty-four worker processes that never touch a + GPU; making torch a module-level import there would cost a CUDA context and + several hundred MB per worker for nothing. + """ + import torch + import torch.nn as nn + + ACT = activations() + + class ResidualBlock(nn.Module): + def __init__(self, hidden, dropout, act): + super().__init__() + self.linear = nn.Linear(hidden, hidden) + self.bn = nn.BatchNorm1d(hidden) + self.act = ACT[act]() + self.drop = nn.Dropout(dropout) + + def forward(self, x): + return x + self.drop(self.act(self.bn(self.linear(x)))) + + class TissueResNet(nn.Module): + def __init__(self, d_in, d_out, n_layers, hidden, dropout, act): + super().__init__() + self.stem = nn.Sequential(nn.Linear(d_in, hidden), + nn.BatchNorm1d(hidden), ACT[act]()) + self.blocks = nn.ModuleList( + [ResidualBlock(hidden, dropout, act) for _ in range(n_layers)]) + self.head = nn.Linear(hidden, d_out) + + def forward(self, x): + x = self.stem(x) + for b in self.blocks: + x = b(x) + return self.head(x) + + class StandardMLP(nn.Module): + def __init__(self, d_in, d_out, n_layers, hidden, dropout, act): + super().__init__() + layers, prev = [], d_in + for _ in range(n_layers): + layers += [nn.Linear(prev, hidden), nn.BatchNorm1d(hidden), + ACT[act](), nn.Dropout(dropout)] + prev = hidden + layers.append(nn.Linear(hidden, d_out)) + self.net = nn.Sequential(*layers) + + def forward(self, x): + return self.net(x) + + class Tabular1DCNN(nn.Module): + def __init__(self, d_in, d_out, n_layers, hidden, dropout, act): + super().__init__() + layers, ch = [], 1 + for _ in range(n_layers): + layers += [nn.Conv1d(ch, hidden, kernel_size=3, padding=1), + nn.BatchNorm1d(hidden), ACT[act](), + nn.Dropout(dropout)] + ch = hidden + self.conv = nn.Sequential(*layers) + self.pool = nn.AdaptiveAvgPool1d(1) + self.head = nn.Linear(hidden, d_out) + + def forward(self, x): + return self.head(self.pool(self.conv(x.unsqueeze(1))).squeeze(2)) + + class FTTransformer(nn.Module): + """ + Feature tokeniser + transformer encoder + CLS head. + + The tokeniser is one affine map per feature, exactly as in the + submitted implementation, but expressed as a single (d_in, d_token) + weight rather than d_in separate Linear modules. For 153 features that + replaces 153 kernel launches per forward pass with one broadcast + multiply. + """ + + def __init__(self, d_in, d_out, n_layers, hidden, dropout, act): + super().__init__() + self.d_token = max(8, (hidden // 8) * 8) # divisible by nhead=8 + nhead = 8 if self.d_token % 8 == 0 else 4 + self.weight = nn.Parameter(torch.empty(d_in, self.d_token)) + self.bias = nn.Parameter(torch.zeros(d_in, self.d_token)) + nn.init.normal_(self.weight, std=self.d_token ** -0.5) + self.cls = nn.Parameter(torch.randn(1, 1, self.d_token) * 0.02) + layer = nn.TransformerEncoderLayer( + d_model=self.d_token, nhead=nhead, + dim_feedforward=self.d_token * 2, dropout=dropout, + activation="gelu", batch_first=True, norm_first=True) + self.encoder = nn.TransformerEncoder(layer, num_layers=n_layers) + self.head = nn.Linear(self.d_token, d_out) + + def forward(self, x): + # (B, F) -> (B, F, d_token) + tokens = x.unsqueeze(-1) * self.weight.unsqueeze(0) + self.bias + cls = self.cls.expand(x.size(0), -1, -1) + out = self.encoder(torch.cat([cls, tokens], dim=1)) + return self.head(out[:, 0, :]) + + class TabNetLite(nn.Module): + def __init__(self, d_in, d_out, n_layers, hidden, dropout, act): + super().__init__() + self.n_steps = max(1, n_layers) + self.bn = nn.BatchNorm1d(d_in) + self.transformers = nn.ModuleList([ + nn.Sequential(nn.Linear(d_in, hidden), nn.BatchNorm1d(hidden), + ACT[act](), nn.Dropout(dropout)) + for _ in range(self.n_steps)]) + self.attentions = nn.ModuleList([ + nn.Sequential(nn.Linear(hidden, d_in), nn.BatchNorm1d(d_in), + nn.Softmax(dim=-1)) + for _ in range(self.n_steps)]) + self.head = nn.Linear(hidden, d_out) + + def forward(self, x): + x = self.bn(x) + prior = torch.ones_like(x) + agg = 0 + rep = self.transformers[0](x) + for step in range(self.n_steps): + mask = self.attentions[step](rep) * prior + prior = prior * (1.0 - mask) + rep = self.transformers[step](x * mask) + agg = agg + rep + return self.head(agg) + + class NeuralDecisionForest(nn.Module): + def __init__(self, d_in, d_out, n_layers, hidden, dropout, act): + super().__init__() + self.n_trees = max(1, hidden // 16) + depth = max(2, n_layers + 1) + self.n_leaves = 2 ** depth + self.trees = nn.ModuleList([ + nn.Sequential(nn.Linear(d_in, self.n_leaves), + nn.Dropout(dropout), nn.Softmax(dim=-1)) + for _ in range(self.n_trees)]) + self.leaves = nn.Parameter( + torch.randn(self.n_trees, self.n_leaves, d_out) * 0.1) + + def forward(self, x): + out = sum(torch.matmul(t(x), self.leaves[i]) + for i, t in enumerate(self.trees)) + return out / self.n_trees + + return {"ResNet": TissueResNet, "MLP": StandardMLP, + "1D_CNN": Tabular1DCNN, "FT_Transformer": FTTransformer, + "TabNet_Lite": TabNetLite, "NODE_Lite": NeuralDecisionForest} + + +@dataclass(frozen=True) +class DeepSpec: + name: str + family: str + notes: str = "" + + +REGISTRY: dict[str, DeepSpec] = { + "MLP": DeepSpec("MLP", "neural", "fully connected with batch norm"), + "ResNet": DeepSpec("ResNet", "neural", "residual blocks"), + "1D_CNN": DeepSpec("1D_CNN", "neural", + "1-D convolution over the feature axis"), + "FT_Transformer": DeepSpec("FT_Transformer", "transformer", + "feature tokeniser + encoder, CLS head"), + "TabNet_Lite": DeepSpec("TabNet_Lite", "transformer", + "sequential attention over features"), + "NODE_Lite": DeepSpec("NODE_Lite", "neural", + "differentiable oblivious decision ensemble"), +} + + +def names() -> list[str]: + return list(REGISTRY) + + +def build(name: str, d_in: int, d_out: int, params: dict): + """Instantiate one architecture from a trial's parameters.""" + mods = _build_modules() + if name not in mods: + raise KeyError(f"unknown architecture {name!r}") + return mods[name](d_in, d_out, params["n_layers"], params["hidden_dim"], + params["dropout"], params.get("activation", "relu")) + + +# ── search space ───────────────────────────────────────────────────────────── + +def suggest(trial) -> dict: + """ + Transcribed from the submitted pipeline's objective, with two changes. + + `num_epochs` is replaced by `max_epochs` plus early stopping: the submitted + search ran every trial for its full sampled epoch count, which on this + dataset means training long past the point validation loss stops improving. + `class_weight` is new, because both targets are imbalanced and an + unweighted loss can reach a good value while never predicting a minority + class. + """ + return { + "n_layers": trial.suggest_int("n_layers", 1, 6), + "hidden_dim": trial.suggest_int("hidden_dim", 64, 512, step=32), + "lr": trial.suggest_float("lr", 1e-4, 1e-2, log=True), + "weight_decay": trial.suggest_float("weight_decay", 1e-5, 1e-2, + log=True), + "dropout": trial.suggest_float("dropout", 0.1, 0.4), + "batch_size": trial.suggest_categorical("batch_size", + [32, 64, 128, 256]), + "activation": trial.suggest_categorical( + "activation", ["relu", "tanh", "GELU", "SELU", "ELU", "SiLU"]), + "optimizer": trial.suggest_categorical( + "optimizer", ["Adam", "AdamW", "RMSprop", "SGD"]), + "max_epochs": trial.suggest_int("max_epochs", 100, 400), + "class_weight": trial.suggest_categorical("class_weight", + [None, "balanced"]), + } + + +# ── training ───────────────────────────────────────────────────────────────── + +def class_weights(y: np.ndarray, n_classes: int, device): + """Inverse-frequency weights, normalised to mean 1.""" + import torch + counts = np.bincount(y, minlength=n_classes).astype(float) + counts[counts == 0] = 1.0 + w = counts.sum() / (n_classes * counts) + return torch.tensor(w / w.mean(), dtype=torch.float32, device=device) + + +def train(name: str, params: dict, Xtr, ytr, Xva, yva, n_classes: int, + device: str, patience: int = 25, report=None, seed: int = SEED): + """ + Train one network with early stopping on validation macro F1. + + `report(epoch, score)` is called after each validation pass and may raise + to abort the trial - that is how Optuna's epoch-level pruning attaches. + + Returns (model, history). The model carries the best weights seen, not the + last, so a run that overfits after epoch 40 still returns epoch 40. + """ + import torch + import torch.nn as nn + from sklearn.metrics import f1_score + + torch.manual_seed(seed) + np.random.seed(seed) + dev = torch.device(device) + + d_in = Xtr.shape[1] + model = build(name, d_in, n_classes, params).to(dev) + opt = optimisers()[params["optimizer"]]( + model.parameters(), lr=params["lr"], + weight_decay=params["weight_decay"]) + + # The whole dataset is ~1.5 MB on device, so it is moved once and never + # transferred again; batching is index slicing on the GPU. + Xtr_t = torch.as_tensor(Xtr, dtype=torch.float32, device=dev) + ytr_t = torch.as_tensor(ytr, dtype=torch.long, device=dev) + Xva_t = torch.as_tensor(Xva, dtype=torch.float32, device=dev) + + weight = (class_weights(ytr, n_classes, dev) + if params.get("class_weight") == "balanced" else None) + criterion = nn.CrossEntropyLoss(weight=weight) + + n = len(ytr_t) + bs = min(params["batch_size"], n) + best_score, best_state, best_epoch, stale = -1.0, None, 0, 0 + history = [] + + for epoch in range(params["max_epochs"]): + model.train() + perm = torch.randperm(n, device=dev) + total = 0.0 + for i in range(0, n, bs): + idx = perm[i:i + bs] + if len(idx) < 2: # BatchNorm needs >1 sample + continue + opt.zero_grad(set_to_none=True) + loss = criterion(model(Xtr_t[idx]), ytr_t[idx]) + if not torch.isfinite(loss): + return model, {"diverged": True, "history": history} + loss.backward() + opt.step() + total += float(loss.detach()) * len(idx) + + model.eval() + with torch.no_grad(): + logits = model(Xva_t) + # Validation LOSS is recorded, not just the score. Figure S10 of + # the supplementary plots training against validation loss, and + # that divergence - not the F1 curve - is what shows a model + # overfitting. Recording only the score would make the figure + # impossible to redraw. + val_loss = float(criterion( + logits, torch.as_tensor(yva, dtype=torch.long, + device=dev)).detach()) + pred = logits.argmax(dim=1).cpu().numpy() + score = float(f1_score(yva, pred, average="macro", zero_division=0)) + history.append({"epoch": epoch, "train_loss": total / n, + "val_loss": val_loss, "val_macro_f1": score}) + + if score > best_score: + best_score, best_epoch, stale = score, epoch, 0 + best_state = {k: v.detach().clone() + for k, v in model.state_dict().items()} + else: + stale += 1 + + if report is not None: + report(epoch, score) + if stale >= patience: + break + + if best_state is not None: + model.load_state_dict(best_state) + return model, {"diverged": False, "best_epoch": best_epoch, + "best_val_macro_f1": best_score, + "epochs_run": len(history), "history": history} + + +def predict_proba(model, X, device: str, batch: int = 4096) -> np.ndarray: + import torch + dev = torch.device(device) + model.eval() + out = [] + with torch.no_grad(): + for i in range(0, len(X), batch): + chunk = torch.as_tensor(X[i:i + batch], dtype=torch.float32, + device=dev) + out.append(torch.softmax(model(chunk), dim=1).cpu().numpy()) + return np.concatenate(out, axis=0) diff --git a/mlate/evaluation.py b/mlate/evaluation.py new file mode 100644 index 0000000000000000000000000000000000000000..acbf5f340090018a705fc6ed1451e9da56c2144c --- /dev/null +++ b/mlate/evaluation.py @@ -0,0 +1,311 @@ +""" +Classification metrics for imbalanced, ordinal targets +====================================================== + +Both targets are imbalanced and ordinal, which rules out reporting accuracy on +its own. Printability class 3 holds 49.9% of the corpus, so a constant +prediction scores 0.499 before any learning takes place; Cell Response class 1 +is rarer still. Every result in step 04 therefore carries, alongside accuracy: + + balanced accuracy mean per-class recall - a model that ignores the minority + classes cannot hide behind the majority + macro F1 unweighted mean of per-class F1, same motivation + weighted F1 F1 weighted by support, for comparability with the + submitted version, which reported it + Cohen's kappa agreement corrected for chance + quadratic kappa the ordinal variant: predicting class 0 when the truth is + class 3 is penalised more heavily than predicting class 2. + Both targets are ordered, so this is the metric that + matches the problem + MCC Matthews correlation, stable under imbalance + +Scores are pooled over folds rather than averaged across them. Out-of-fold +predictions are concatenated and scored once, so every sample contributes +equally regardless of which fold it landed in; the mean-of-folds alternative +gives small folds disproportionate weight and has visibly higher variance. The +per-fold values are retained so that dispersion can still be reported. +""" + +from __future__ import annotations + +import numpy as np +import pandas as pd +from sklearn.metrics import (accuracy_score, balanced_accuracy_score, + cohen_kappa_score, confusion_matrix, f1_score, + log_loss, matthews_corrcoef, + precision_recall_fscore_support, roc_auc_score) + +from mlate import config as cfg + +N_BOOTSTRAP = 1000 + + +def scores(y_true, y_pred, proba=None, labels=None) -> dict: + """ + The metric panel for one set of predictions. + + `proba` is an (n_samples, n_classes) array of predicted probabilities + aligned to `labels`. When it is supplied the probability-based metrics are + added; when a model has no predict_proba they come back as NaN rather than + being silently omitted, so every model occupies the same columns. + """ + y_true = np.asarray(y_true) + y_pred = np.asarray(y_pred) + out = { + "accuracy": float(accuracy_score(y_true, y_pred)), + "balanced_accuracy": float(balanced_accuracy_score(y_true, y_pred)), + "macro_precision": float(precision_recall_fscore_support( + y_true, y_pred, average="macro", zero_division=0)[0]), + "macro_recall": float(precision_recall_fscore_support( + y_true, y_pred, average="macro", zero_division=0)[1]), + "weighted_precision": float(precision_recall_fscore_support( + y_true, y_pred, average="weighted", zero_division=0)[0]), + "weighted_recall": float(precision_recall_fscore_support( + y_true, y_pred, average="weighted", zero_division=0)[1]), + "macro_f1": float(f1_score(y_true, y_pred, average="macro", + zero_division=0)), + "weighted_f1": float(f1_score(y_true, y_pred, average="weighted", + zero_division=0)), + "kappa": float(cohen_kappa_score(y_true, y_pred)), + "quadratic_kappa": float(cohen_kappa_score(y_true, y_pred, + weights="quadratic")), + "mcc": float(matthews_corrcoef(y_true, y_pred)), + "specificity_macro": _specificity(y_true, y_pred, labels), + } + out.update(_probability_scores(y_true, proba, labels)) + return out + + +def _specificity(y_true, y_pred, labels=None) -> float: + """ + Mean per-class true-negative rate. + + Macro recall is the mean sensitivity; reporting its complement alongside is + what lets a reader see that a model achieving high specificity on a rare + class may be doing so by never predicting it. + """ + labels = list(labels) if labels is not None else sorted(set(np.asarray(y_true))) + cm = confusion_matrix(y_true, y_pred, labels=labels) + total = cm.sum() + out = [] + for i in range(len(labels)): + tp = cm[i, i] + fn = cm[i, :].sum() - tp + fp = cm[:, i].sum() - tp + tn = total - tp - fn - fp + out.append(tn / (tn + fp) if (tn + fp) else np.nan) + return float(np.nanmean(out)) + + +def _probability_scores(y_true, proba, labels) -> dict: + """ROC-AUC, log loss and the multiclass Brier score, or NaN throughout.""" + blank = {"roc_auc_macro_ovr": np.nan, "roc_auc_weighted_ovr": np.nan, + "log_loss": np.nan, "brier_multiclass": np.nan} + if proba is None or labels is None: + return blank + proba = np.asarray(proba, dtype=float) + labels = list(labels) + if proba.ndim != 2 or proba.shape[1] != len(labels) or np.isnan(proba).any(): + return blank + + # Renormalise defensively; some estimators return rows that do not sum to 1. + rows = proba.sum(axis=1, keepdims=True) + proba = np.divide(proba, np.where(rows == 0, 1.0, rows)) + + present = sorted(set(np.asarray(y_true))) + out = dict(blank) + try: + # One-vs-rest AUC is undefined for a class absent from y_true, so the + # score is restricted to the classes actually observed and renormalised. + keep = [labels.index(c) for c in present] + if len(present) >= 2: + sub = proba[:, keep] + sub = sub / np.where(sub.sum(axis=1, keepdims=True) == 0, 1.0, + sub.sum(axis=1, keepdims=True)) + kw = dict(labels=present, multi_class="ovr") + out["roc_auc_macro_ovr"] = float( + roc_auc_score(y_true, sub, average="macro", **kw)) + out["roc_auc_weighted_ovr"] = float( + roc_auc_score(y_true, sub, average="weighted", **kw)) + except Exception: + pass + try: + out["log_loss"] = float(log_loss(y_true, proba, labels=labels)) + except Exception: + pass + try: + onehot = np.zeros_like(proba) + index = {c: i for i, c in enumerate(labels)} + for r, c in enumerate(np.asarray(y_true)): + onehot[r, index[c]] = 1.0 + out["brier_multiclass"] = float(((proba - onehot) ** 2).sum(axis=1).mean()) + except Exception: + pass + return out + + +def per_class(y_true, y_pred, labels) -> pd.DataFrame: + """Precision, recall, F1 and support for every class, including absent ones.""" + p, r, f, s = precision_recall_fscore_support( + y_true, y_pred, labels=labels, zero_division=0) + predicted = pd.Series(y_pred).value_counts() + return pd.DataFrame({ + "class": labels, "precision": p, "recall": r, "f1": f, + "support": s, + "n_predicted": [int(predicted.get(c, 0)) for c in labels], + }) + + +def confusion(y_true, y_pred, labels) -> pd.DataFrame: + m = confusion_matrix(y_true, y_pred, labels=labels) + return pd.DataFrame(m, index=[f"true_{c}" for c in labels], + columns=[f"pred_{c}" for c in labels]) + + +def bootstrap_ci(y_true, y_pred, metric: str = "macro_f1", + n: int = N_BOOTSTRAP, alpha: float = 0.05, + seed: int = cfg.RANDOM_STATE) -> tuple[float, float]: + """ + Percentile confidence interval by resampling predictions. + + This is an interval on the metric given these predictions, not on the whole + modelling procedure - refitting inside every replicate would be the stricter + construction and is not affordable across 30 models and three protocols. It + is therefore an optimistic interval and is labelled as such wherever it is + reported. + """ + y_true = np.asarray(y_true) + y_pred = np.asarray(y_pred) + rng = np.random.default_rng(seed) + + # Compute only the requested metric. Calling scores() here would evaluate + # the whole panel - including ROC-AUC and a confusion matrix - a thousand + # times per cell, which dominates the entire reporting step. + single = { + "macro_f1": lambda t, p_: f1_score(t, p_, average="macro", + zero_division=0), + "weighted_f1": lambda t, p_: f1_score(t, p_, average="weighted", + zero_division=0), + "accuracy": accuracy_score, + "balanced_accuracy": balanced_accuracy_score, + "mcc": matthews_corrcoef, + "kappa": cohen_kappa_score, + "quadratic_kappa": lambda t, p_: cohen_kappa_score(t, p_, + weights="quadratic"), + }.get(metric) + if single is None: + single = lambda t, p_: scores(t, p_)[metric] + + out = [] + for _ in range(n): + idx = rng.integers(0, len(y_true), len(y_true)) + if len(np.unique(y_true[idx])) < 2: + continue + out.append(float(single(y_true[idx], y_pred[idx]))) + if not out: + return (np.nan, np.nan) + return (float(np.percentile(out, 100 * alpha / 2)), + float(np.percentile(out, 100 * (1 - alpha / 2)))) + + +TASK_LABELS = {"printability": [0, 1, 2, 3], + "cell_response": [1, 2, 3, 4, 5], + "cell_response_cellular": [2, 3, 4, 5]} + + +def score_predictions(preds: pd.DataFrame, + bootstrap: bool = True) -> pd.DataFrame: + """ + Score a stored prediction table, whatever produced it. + + Conventional models, deep architectures and the foundation models all write + the same schema, so they are scored here by the same code rather than by + three near-copies that could drift. Grouping keys are model, task, + protocol, selection and split. + + `split` is part of the key and never collapsed: a training score is a + diagnostic and must stay separately labelled, so that no downstream table + can accidentally present one as a result. + + Bootstrap intervals are computed on the test partition only. Under the + hold-out design the reported number rests on a single 530- or 574-row + partition, which makes an explicit interval more necessary than it was + under cross-validation - Referee 1's comment 2 asks for exactly this. + """ + rows = [] + keys = ["model", "task", "protocol", "selection", "split"] + keys = [k for k in keys if k in preds.columns] + for values, g in preds.groupby(keys, sort=False): + record = dict(zip(keys, values if isinstance(values, tuple) + else (values,))) + labels = TASK_LABELS[record["task"]] + pcols = [f"p_{c}" for c in labels] + proba = None + if all(c in g.columns for c in pcols): + arr = g[pcols].to_numpy(dtype=float) + proba = None if np.isnan(arr).any() else arr + record["n_scored"] = int(len(g)) + record.update(scores(g["y_true"], g["y_pred"], proba, labels)) + if bootstrap and record.get("split") == "test": + for metric in ("weighted_f1", "macro_f1"): + lo, hi = bootstrap_ci(g["y_true"], g["y_pred"], metric) + record[f"{metric}_lo"] = lo + record[f"{metric}_hi"] = hi + rows.append(record) + return pd.DataFrame(rows) + + +def train_test_gap(board: pd.DataFrame, + metric: str = "weighted_f1") -> pd.DataFrame: + """ + Training minus test score for every model, as an overfitting diagnostic. + + Under the study-grouped protocol this separates two failures that look + identical in a test column alone: a model that could not learn the task at + all, and one that learned study-specific structure which does not transfer. + They call for opposite remedies - better features versus more studies - so + the distinction is worth a column. + """ + keys = [k for k in ("model", "task", "protocol", "selection") + if k in board.columns] + wide = board.pivot_table(index=keys, columns="split", values=metric) + if not {"train", "test"} <= set(wide.columns): + return pd.DataFrame() + wide = wide.reset_index() + wide[f"{metric}_gap"] = wide["train"] - wide["test"] + return wide.rename(columns={"train": f"{metric}_train", + "test": f"{metric}_test"}) + + +def summarise(frame: pd.DataFrame, labels) -> pd.DataFrame: + """ + Collapse per-sample out-of-fold predictions into one row per + model x task x protocol. + + `frame` must carry columns: model, task, protocol, fold, y_true, y_pred. + """ + rows = [] + keys = ["model", "task", "protocol"] + pcols = [f"p_{c}" for c in labels] + + def _proba(block): + if not all(c in block.columns for c in pcols): + return None + arr = block[pcols].to_numpy(dtype=float) + return None if np.isnan(arr).any() else arr + + for (model, task, protocol), g in frame.groupby(keys, sort=False): + pooled = scores(g["y_true"], g["y_pred"], _proba(g), labels) + by_fold = pd.DataFrame([ + scores(h["y_true"], h["y_pred"], _proba(h), labels) + for _, h in g.groupby("fold")]) + lo, hi = bootstrap_ci(g["y_true"], g["y_pred"], "macro_f1") + rows.append({ + "model": model, "task": task, "protocol": protocol, + "n_scored": int(len(g)), "n_folds": int(g["fold"].nunique()), + **pooled, + "macro_f1_lo": lo, "macro_f1_hi": hi, + "accuracy_sd": float(by_fold["accuracy"].std(ddof=1)), + "macro_f1_sd": float(by_fold["macro_f1"].std(ddof=1)), + }) + return pd.DataFrame(rows) diff --git a/mlate/imputation.py b/mlate/imputation.py new file mode 100644 index 0000000000000000000000000000000000000000..0b2a89edb55c94ee5f33b8d6f06e1cb58c4018ff --- /dev/null +++ b/mlate/imputation.py @@ -0,0 +1,372 @@ +""" +Tiered imputation +================= + +Blank does not mean the same thing in every column of this dataset, so a +single imputer would be wrong. Four rules, each matched to what a missing +value actually signifies: + +Syringe / Substrate Temperature + Missing because the study printed at ambient and did not think it worth + reporting. The missingness is informative, so it is filled with a constant + (22 C, the modal reported value) plus a '_reported' indicator. A + model-based imputer would be actively harmful here: it learns from the + rows that DO report a temperature, which are exactly the deliberately + heated and cooled ones, and would push ambient samples toward controlled + values. + +Pressure, speeds, crosslinking durations, nozzle diameter + No physical default exists; blank genuinely means unknown. Model-based + imputation plus a '_reported' indicator. + +Biomaterial concentrations + Per the dataset construction rule, absent = 0 and unreported = blank. A + blank therefore means the material IS in the formulation at an unstated + concentration, so it takes the median concentration of that material + across samples where it is present. Filling 0 would silently delete an + ingredient. + +Cell density + Strongly cell-line specific, so the median for the same cell line is used + where available, falling back to the global median. + +Every estimator here learns from training rows only. Under grouped +cross-validation the whole pipeline is refitted inside each fold, so no +information crosses the train/test boundary through an imputed value. +""" + +from __future__ import annotations + +import warnings + +import numpy as np +import pandas as pd +from sklearn.exceptions import ConvergenceWarning +from sklearn.base import BaseEstimator, TransformerMixin +from sklearn.experimental import enable_iterative_imputer # noqa: F401 +from sklearn.impute import IterativeImputer +from sklearn.ensemble import ExtraTreesRegressor + +warnings.filterwarnings( + 'ignore', message='.*Early stopping criterion not reached.*', + category=ConvergenceWarning) + +from mlate import config as cfg + + +class AmbientFill(BaseEstimator, TransformerMixin): + """Constant fill for informatively-missing columns, plus an indicator.""" + + def __init__(self, value: float = cfg.AMBIENT_TEMPERATURE_C, + add_indicator: bool = True): + self.value = value + self.add_indicator = add_indicator + + def fit(self, X, y=None): + self.feature_names_in_ = list(pd.DataFrame(X).columns) + return self + + def transform(self, X): + X = pd.DataFrame(X).copy() + out = X.fillna(self.value) + if self.add_indicator: + for c in X.columns: + out[f"{c} [reported]"] = X[c].notna().astype(float) + return out.to_numpy(dtype=float) + + def get_feature_names_out(self, input_features=None): + names = list(input_features if input_features is not None + else self.feature_names_in_) + if self.add_indicator: + names += [f"{c} [reported]" for c in names] + return np.asarray(names, dtype=object) + + +class PresentMedianFill(BaseEstimator, TransformerMixin): + """ + Biomaterial rule: blank means present-but-unreported. + + Fills with the median over training samples where the material is present + (non-zero). Materials never present in training fall back to 0. + """ + + def fit(self, X, y=None): + X = pd.DataFrame(X) + self.feature_names_in_ = list(X.columns) + self.fill_ = {} + for c in X.columns: + present = X[c][X[c].fillna(0) != 0] + self.fill_[c] = float(present.median()) if len(present) else 0.0 + return self + + def transform(self, X): + X = pd.DataFrame(X).copy() + for c in X.columns: + X[c] = X[c].fillna(self.fill_.get(c, 0.0)) + return X.to_numpy(dtype=float) + + def get_feature_names_out(self, input_features=None): + return np.asarray(input_features if input_features is not None + else self.feature_names_in_, dtype=object) + + +def _make_estimator(name: str, seed: int): + """Regressor driving IterativeImputer. Benchmarked in + preprocessing/validate_imputation.py.""" + if name == "extra_trees": + return ExtraTreesRegressor(n_estimators=cfg.IMPUTER_N_ESTIMATORS, + max_depth=12, + n_jobs=cfg.N_JOBS, random_state=seed) + if name == "xgboost": + from xgboost import XGBRegressor + return XGBRegressor(n_estimators=cfg.IMPUTER_N_ESTIMATORS * 3, + max_depth=6, learning_rate=0.1, + tree_method="hist", n_jobs=cfg.N_JOBS, + verbosity=0, random_state=seed) + raise ValueError(f"unknown imputer estimator: {name}") + + +class MedianFill(BaseEstimator, TransformerMixin): + """ + Column median plus a [reported] indicator. The default for printing + parameters. + + This looks like the naive choice and is in fact the evidence-based one. + Under study-grouped masking - the honest test, matching how the pipeline + actually runs - the median beats every multivariate imputer tried + (validate_imputation.py: nMAE 1.30 against 1.54 for iterative ExtraTrees, + 1.90 for XGBoost, 1.99 for KNN), and every method including this one has a + negative R^2. Printing parameters simply are not predictable across studies: + knowing a formulation's composition tells you very little about the pressure + or nozzle speed some other laboratory chose. + + Sophisticated imputation looked far better under ungrouped masking only + because it was recovering values from the same paper's other rows. What + carries real information here is not the filled value but the indicator + saying the value was never reported. + """ + + def __init__(self, add_indicator: bool = True): + self.add_indicator = add_indicator + + def fit(self, X, y=None): + X = pd.DataFrame(X) + self.feature_names_in_ = list(X.columns) + self.fill_ = {c: (float(X[c].median()) if X[c].notna().any() else 0.0) + for c in X.columns} + return self + + def transform(self, X): + X = pd.DataFrame(X) + out = X.copy() + for c in X.columns: + out[c] = pd.to_numeric(out[c], errors="coerce").fillna(self.fill_[c]) + if self.add_indicator: + for c in X.columns: + out[f"{c} [reported]"] = X[c].notna().astype(float) + return out.to_numpy(dtype=float) + + def get_feature_names_out(self, input_features=None): + names = list(input_features if input_features is not None + else self.feature_names_in_) + if self.add_indicator: + names += [f"{c} [reported]" for c in names] + return np.asarray(names, dtype=object) + + +class ModelBasedFill(BaseEstimator, TransformerMixin): + """ + Iterative (multivariate) imputation plus a reported-indicator. + + Retained for the benchmark and as a config option, but NOT the default: + see MedianFill for why it loses once study leakage is removed. It is also + the reason the fitted pipeline was hundreds of megabytes, since + IterativeImputer keeps every sub-model to replay at transform time. + """ + + def __init__(self, add_indicator: bool = True, + random_state: int = cfg.RANDOM_STATE, + max_iter: int = cfg.IMPUTER_MAX_ITER, + estimator: str = cfg.IMPUTER_ESTIMATOR): + self.add_indicator = add_indicator + self.random_state = random_state + self.max_iter = max_iter + self.estimator = estimator + + def fit(self, X, y=None): + X = pd.DataFrame(X) + self.feature_names_in_ = list(X.columns) + self.imputer_ = IterativeImputer( + estimator=_make_estimator(self.estimator, self.random_state), + max_iter=self.max_iter, random_state=self.random_state, + initial_strategy="median", skip_complete=True, + ) + self.imputer_.fit(X) + return self + + def transform(self, X): + X = pd.DataFrame(X) + filled = pd.DataFrame(self.imputer_.transform(X), columns=X.columns, + index=X.index) + if self.add_indicator: + for c in X.columns: + filled[f"{c} [reported]"] = X[c].notna().astype(float).values + return filled.to_numpy(dtype=float) + + def get_feature_names_out(self, input_features=None): + names = list(input_features if input_features is not None + else self.feature_names_in_) + if self.add_indicator: + names += [f"{c} [reported]" for c in names] + return np.asarray(names, dtype=object) + + +class StoredFill(BaseEstimator, TransformerMixin): + """ + Fill from pre-computed per-column values, plus the same [reported] + indicator the training imputers emit. + + This is the deployment counterpart of ModelBasedFill. A fitted + IterativeImputer retains every sub-model so it can replay the imputation + sequence, which makes it hundreds of megabytes - fine as a training-time + object, impossible to ship. At inference the user supplies the formulation, + so there is normally nothing to impute at all; this transformer exists only + to keep the feature space identical and to cover a field left blank. + + The stored values are the medians of what the training imputer actually + produced, so a blank field lands where the heavy imputer would have put it. + """ + + def __init__(self, values: dict[str, float], add_indicator: bool = True): + self.values = values + self.add_indicator = add_indicator + + def fit(self, X, y=None): + self.feature_names_in_ = list(pd.DataFrame(X).columns) + return self + + def transform(self, X): + X = pd.DataFrame(X) + out = X.copy() + for c in X.columns: + out[c] = pd.to_numeric(out[c], errors="coerce").fillna( + self.values.get(c, 0.0)) + if self.add_indicator: + for c in X.columns: + out[f"{c} [reported]"] = X[c].notna().astype(float) + return out.to_numpy(dtype=float) + + def get_feature_names_out(self, input_features=None): + names = list(input_features if input_features is not None + else self.feature_names_in_) + if self.add_indicator: + names += [f"{c} [reported]" for c in names] + return np.asarray(names, dtype=object) + + +class StoredGroupFill(BaseEstimator, TransformerMixin): + """ + Deployment counterpart of CellDensityFill. + + Carries the per-cell-line medians learned during training so a blank + density at inference lands on the same value the training pipeline would + have used. Filling with a single global median instead would make the + shipped preprocessor disagree with the one the models were trained on. + """ + + def __init__(self, by_group: dict, fallback: float): + self.by_group = by_group + self.fallback = fallback + + def fit(self, X, y=None): + self.feature_names_in_ = list(pd.DataFrame(X).columns) + return self + + def transform(self, X): + X = pd.DataFrame(X) + group, value = X.columns[0], X.columns[1] + v = pd.to_numeric(X[value], errors="coerce") + mapped = X[group].map(self.by_group).astype(float) + return v.fillna(mapped).fillna(self.fallback).to_numpy( + dtype=float).reshape(-1, 1) + + def get_feature_names_out(self, input_features=None): + return np.asarray([cfg.CELL_COLS[1]], dtype=object) + + +class CellDensityFill(BaseEstimator, TransformerMixin): + """ + Median density of the same cell line, falling back to the global median. + + Expects two columns, in order: [cell line, cell density]. + """ + + def fit(self, X, y=None): + X = pd.DataFrame(X) + self.feature_names_in_ = list(X.columns) + line, dens = X.columns[0], X.columns[1] + d = pd.to_numeric(X[dens], errors="coerce") + self.by_line_ = d.groupby(X[line]).median().to_dict() + self.global_ = float(d.median()) if d.notna().any() else 0.0 + return self + + def transform(self, X): + X = pd.DataFrame(X) + line, dens = X.columns[0], X.columns[1] + d = pd.to_numeric(X[dens], errors="coerce") + fill = X[line].map(self.by_line_).astype(float) + return d.fillna(fill).fillna(self.global_).to_numpy( + dtype=float).reshape(-1, 1) + + def get_feature_names_out(self, input_features=None): + return np.asarray([cfg.CELL_COLS[1]], dtype=object) + + +def fill_within_study(df: pd.DataFrame, columns, group: str = "DOI" + ) -> tuple[pd.DataFrame, pd.DataFrame]: + """ + Tier 1 of the two-tier fill: copy a printing parameter from the same study. + + Printing parameters are chosen once per publication and reused for every + formulation in it - 87-98% of each column's variance lies between studies, + and 46-94% of studies report a single constant value. So where a study + reports a parameter in some of its rows and leaves it blank in others, the + reported value is direct evidence for the blank ones, not a guess. Filling + it is more accurate than any model, and it is auditable in a way a model is + not. + + This covers 585 of the 6,152 missing cells (9.5%). The other 90.5% sit in + studies that never reported the parameter at all, so no within-study + evidence exists and they fall through to tier 2 - the global median or the + 22 C ambient constant - in build_preprocessor(). + + Used for the published feature matrix and the clustering inputs. It is + deliberately NOT part of the supervised pipeline, where the imputer is + refitted inside each fold; see 02_preprocessing/METHODS.md section 4. + + Returns the filled frame and a per-cell audit of what tier 1 changed. + """ + out = df.copy() + records = [] + for col in columns.print_params: + missing = out[col].isna() + if not missing.any(): + continue + study_median = out.groupby(group)[col].transform("median") + filled = missing & study_median.notna() + if not filled.any(): + continue + records.append(pd.DataFrame({ + "row": out.index[filled], + group: out.loc[filled, group].to_numpy(), + "column": col, + "filled_value": study_median[filled].to_numpy(), + "n_reported_in_study": out.loc[filled, group].map( + out.groupby(group)[col].count()).to_numpy(), + })) + out.loc[filled, col] = study_median[filled] + audit = (pd.concat(records, ignore_index=True) if records + else pd.DataFrame(columns=["row", group, "column", + "filled_value", + "n_reported_in_study"])) + return out, audit diff --git a/mlate/models.py b/mlate/models.py new file mode 100644 index 0000000000000000000000000000000000000000..a245ea8727426764efafaeaeee50e6ed862e0785 --- /dev/null +++ b/mlate/models.py @@ -0,0 +1,299 @@ +""" +The classifier zoo +================== + +One registry, so that every step of the paper draws its models from the same +place and a model cannot be configured differently in two scripts. + +Each entry carries what the runner needs to schedule it, not just how to build +it: whether it parallelises internally (so the outer loop does not oversubscribe +the CPU), whether it wants a GPU, whether it can produce calibrated +probabilities, and roughly how expensive it is. Getting this wrong is the +difference between a benchmark that finishes and one that thrashes. + +All models receive the same input: the fold-fitted preprocessor's output, which +is dense, numeric, MinMax scaled to [0, 1] and free of NaN. Scaling is therefore +not a per-model concern, which is why no entry carries a scaler of its own. + +Two dummy baselines are included deliberately and are not padding. On a task +where the majority class holds 49.9% of the corpus, an accuracy of 0.55 means +nothing until it is placed against the 0.499 a constant prediction achieves, and +the grouped protocol makes that comparison sharper still. +""" + +from __future__ import annotations + +import warnings +from dataclasses import dataclass, field +from typing import Callable + +import numpy as np +from sklearn.discriminant_analysis import (LinearDiscriminantAnalysis, + QuadraticDiscriminantAnalysis) +from sklearn.dummy import DummyClassifier +from sklearn.ensemble import (AdaBoostClassifier, BaggingClassifier, + ExtraTreesClassifier, GradientBoostingClassifier, + HistGradientBoostingClassifier, + RandomForestClassifier, StackingClassifier, + VotingClassifier) +from sklearn.linear_model import (LogisticRegression, PassiveAggressiveClassifier, + Perceptron, RidgeClassifier, SGDClassifier) +from sklearn.naive_bayes import BernoulliNB, GaussianNB +from sklearn.neighbors import KNeighborsClassifier, NearestCentroid +from sklearn.neural_network import MLPClassifier +from sklearn.svm import SVC, LinearSVC +from sklearn.tree import DecisionTreeClassifier, ExtraTreeClassifier + +from mlate import config as cfg + +SEED = cfg.RANDOM_STATE + + +@dataclass(frozen=True) +class Spec: + """How to build one classifier and how to schedule it.""" + name: str + family: str + build: Callable[..., object] + threaded: bool = False # uses n_jobs / OMP internally + gpu: bool = False # can be placed on a CUDA device + proba: bool = True # exposes predict_proba + cost: str = "low" # low | medium | high, for scheduling order + notes: str = "" + + +def _zoo() -> list[Spec]: + S = Spec + return [ + # ── baselines ─────────────────────────────────────────────────────── + S("Dummy (majority)", "baseline", + lambda **k: DummyClassifier(strategy="most_frequent"), + notes="predicts the training majority class for every sample"), + S("Dummy (stratified)", "baseline", + lambda **k: DummyClassifier(strategy="stratified", random_state=SEED), + notes="samples from the training class distribution"), + + # ── linear ────────────────────────────────────────────────────────── + S("Logistic Regression", "linear", + lambda n_jobs=1, **k: LogisticRegression( + max_iter=3000, C=1.0, n_jobs=n_jobs, random_state=SEED), + threaded=True), + S("Logistic Regression (balanced)", "linear", + lambda n_jobs=1, **k: LogisticRegression( + max_iter=3000, C=1.0, class_weight="balanced", n_jobs=n_jobs, + random_state=SEED), + threaded=True, + notes="class_weight balanced; the minority classes are small"), + S("Ridge Classifier", "linear", + lambda **k: RidgeClassifier(alpha=1.0, random_state=SEED), + proba=False), + S("SGD (hinge)", "linear", + lambda **k: SGDClassifier(loss="hinge", max_iter=3000, tol=1e-4, + random_state=SEED), + proba=False), + S("Passive Aggressive", "linear", + lambda **k: PassiveAggressiveClassifier(max_iter=3000, tol=1e-4, + random_state=SEED), + proba=False), + S("Perceptron", "linear", + lambda **k: Perceptron(max_iter=3000, tol=1e-4, random_state=SEED), + proba=False), + + # ── discriminant and probabilistic ────────────────────────────────── + S("Linear Discriminant", "discriminant", + lambda **k: LinearDiscriminantAnalysis(solver="lsqr", + shrinkage="auto")), + S("Quadratic Discriminant", "discriminant", _qda, + notes="PCA to 40 components first; QDA estimates a covariance per " + "class and the smallest class has fewer samples than the 153 " + "features, which makes those covariances singular"), + S("Gaussian Naive Bayes", "naive_bayes", lambda **k: GaussianNB()), + S("Bernoulli Naive Bayes", "naive_bayes", + lambda **k: BernoulliNB(), + notes="binarises at 0.5; suits the sparse biomaterial block"), + + # ── instance based ────────────────────────────────────────────────── + S("k-Nearest Neighbours", "neighbours", + lambda n_jobs=1, **k: KNeighborsClassifier(n_neighbors=5, + n_jobs=n_jobs), + threaded=True), + S("k-NN (distance weighted)", "neighbours", + lambda n_jobs=1, **k: KNeighborsClassifier( + n_neighbors=15, weights="distance", n_jobs=n_jobs), + threaded=True), + S("Nearest Centroid", "neighbours", + lambda **k: NearestCentroid(), proba=False), + + # ── kernel machines ───────────────────────────────────────────────── + S("SVM (RBF)", "svm", + lambda **k: SVC(kernel="rbf", C=10.0, gamma="scale", + probability=True, random_state=SEED), + cost="high"), + S("SVM (polynomial)", "svm", + lambda **k: SVC(kernel="poly", degree=3, C=10.0, gamma="scale", + probability=True, random_state=SEED), + cost="high"), + S("Linear SVM", "svm", + lambda **k: LinearSVC(C=1.0, max_iter=5000, dual="auto", + random_state=SEED), + proba=False), + + # ── single trees ──────────────────────────────────────────────────── + S("Decision Tree", "tree", + lambda **k: DecisionTreeClassifier(max_depth=12, min_samples_leaf=3, + random_state=SEED)), + S("Extra Tree", "tree", + lambda **k: ExtraTreeClassifier(max_depth=12, min_samples_leaf=3, + random_state=SEED)), + + # ── bagging ensembles ─────────────────────────────────────────────── + S("Random Forest", "bagging", + lambda n_jobs=1, **k: RandomForestClassifier( + n_estimators=500, min_samples_leaf=2, n_jobs=n_jobs, + random_state=SEED), + threaded=True, cost="medium"), + S("Random Forest (balanced)", "bagging", + lambda n_jobs=1, **k: RandomForestClassifier( + n_estimators=500, min_samples_leaf=2, + class_weight="balanced_subsample", n_jobs=n_jobs, + random_state=SEED), + threaded=True, cost="medium"), + S("Extra Trees", "bagging", + lambda n_jobs=1, **k: ExtraTreesClassifier( + n_estimators=500, min_samples_leaf=2, n_jobs=n_jobs, + random_state=SEED), + threaded=True, cost="medium"), + S("Bagged Trees", "bagging", + lambda n_jobs=1, **k: BaggingClassifier( + estimator=DecisionTreeClassifier(max_depth=12, + random_state=SEED), + n_estimators=200, n_jobs=n_jobs, random_state=SEED), + threaded=True, cost="medium"), + + # ── boosting ──────────────────────────────────────────────────────── + S("AdaBoost", "boosting", + lambda **k: AdaBoostClassifier(n_estimators=300, learning_rate=0.5, + random_state=SEED), + cost="medium"), + S("Gradient Boosting", "boosting", + lambda **k: GradientBoostingClassifier( + n_estimators=300, learning_rate=0.1, max_depth=3, + random_state=SEED), + cost="high"), + S("Hist Gradient Boosting", "boosting", + lambda **k: HistGradientBoostingClassifier( + max_iter=400, learning_rate=0.08, early_stopping=True, + validation_fraction=0.15, random_state=SEED), + threaded=True, cost="medium"), + S("XGBoost", "boosting", _xgboost, threaded=True, gpu=True, + cost="medium"), + S("LightGBM", "boosting", _lightgbm, threaded=True, cost="medium"), + S("CatBoost", "boosting", _catboost, threaded=True, cost="medium", + notes="ordered boosting; symmetric trees"), + + # ── neural ────────────────────────────────────────────────────────── + S("MLP (256-128)", "neural", + lambda **k: MLPClassifier( + hidden_layer_sizes=(256, 128), alpha=1e-3, max_iter=600, + early_stopping=True, n_iter_no_change=25, random_state=SEED), + threaded=True, cost="medium"), + + # ── meta-ensembles ────────────────────────────────────────────────── + S("Soft Voting (RF+XGB+LR)", "meta", _voting, threaded=True, + cost="high", + notes="soft vote over a bagging, a boosting and a linear learner"), + S("Stacking (RF+XGB+LR -> LR)", "meta", _stacking, threaded=True, + cost="high", + notes="5-fold internal stacking, logistic meta-learner"), + ] + + +# ── library-backed builders, kept out of the table for readability ────────── + +def _xgboost(n_jobs: int = 1, device: str | None = None, **kwargs): + from xgboost import XGBClassifier + params = dict(n_estimators=600, learning_rate=0.06, max_depth=6, + subsample=0.85, colsample_bytree=0.85, reg_lambda=1.0, + tree_method="hist", n_jobs=n_jobs, random_state=SEED, + eval_metric="mlogloss") + if device: + params["device"] = device + return XGBClassifier(**params) + + +def _lightgbm(n_jobs: int = 1, **kwargs): + from lightgbm import LGBMClassifier + return LGBMClassifier(n_estimators=600, learning_rate=0.06, max_depth=-1, + num_leaves=63, subsample=0.85, subsample_freq=1, + colsample_bytree=0.85, n_jobs=n_jobs, + random_state=SEED, verbose=-1) + + +def _qda(**kwargs): + from sklearn.decomposition import PCA + from sklearn.pipeline import Pipeline + return Pipeline([ + ("pca", PCA(n_components=40, random_state=SEED)), + ("qda", QuadraticDiscriminantAnalysis(reg_param=0.3)), + ]) + + +def _catboost(n_jobs: int = 1, **kwargs): + from catboost import CatBoostClassifier + return CatBoostClassifier(iterations=600, learning_rate=0.06, depth=6, + l2_leaf_reg=3.0, thread_count=n_jobs, + random_seed=SEED, verbose=0, + allow_writing_files=False) + + +def _bases(n_jobs: int = 1): + return [ + ("rf", RandomForestClassifier(n_estimators=300, min_samples_leaf=2, + n_jobs=n_jobs, random_state=SEED)), + ("xgb", _xgboost(n_jobs=n_jobs)), + ("lr", LogisticRegression(max_iter=3000, random_state=SEED)), + ] + + +def _voting(n_jobs: int = 1, **kwargs): + return VotingClassifier(estimators=_bases(n_jobs), voting="soft", + n_jobs=1) + + +def _stacking(n_jobs: int = 1, **kwargs): + return StackingClassifier( + estimators=_bases(n_jobs), + final_estimator=LogisticRegression(max_iter=3000, random_state=SEED), + cv=5, n_jobs=1, passthrough=False) + + +# ── public API ────────────────────────────────────────────────────────────── + +REGISTRY: dict[str, Spec] = {s.name: s for s in _zoo()} + + +def names(exclude_baselines: bool = False) -> list[str]: + return [n for n, s in REGISTRY.items() + if not (exclude_baselines and s.family == "baseline")] + + +def build(name: str, n_jobs: int = 1, device: str | None = None): + """Instantiate a fresh, unfitted estimator.""" + spec = REGISTRY[name] + kwargs = {} + if spec.threaded: + kwargs["n_jobs"] = n_jobs + if spec.gpu and device: + kwargs["device"] = device + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + return spec.build(**kwargs) + + +def summary(): + import pandas as pd + return pd.DataFrame([{ + "model": s.name, "family": s.family, "threaded": s.threaded, + "gpu": s.gpu, "predict_proba": s.proba, "cost": s.cost, + "notes": s.notes, + } for s in REGISTRY.values()]) diff --git a/mlate/optimize.py b/mlate/optimize.py new file mode 100644 index 0000000000000000000000000000000000000000..b4a826978519dcf6082b04390c1b9f10ea20fa82 --- /dev/null +++ b/mlate/optimize.py @@ -0,0 +1,353 @@ +""" +Scaffold optimisation +===================== + +Searches the formulation space for the composition and printing conditions that +maximise WSSQ, using the deployed classifiers as the objective. + +Extracted from the Streamlit application deliberately. In the previous version +the objective function lived inside the interface, which meant it could not be +run headlessly, could not be tested, and could not produce a result the +manuscript was able to report. Here the interface calls this module rather than +containing it. + +How a classifier becomes a continuous objective +----------------------------------------------- +Printability and Cell Response are ordinal classes, so a classifier returns a +probability vector rather than a number, and an optimiser handed the arg-max +would be climbing a step function with four or five levels. Instead each model's +full probability vector is collapsed to its EXPECTED CLASS VALUE, + + E[class] = sum_k P(class = k) * label_k + +which is continuous, respects the ordinal spacing of the labels, and carries the +model's uncertainty: a formulation the model is unsure about lands between +classes rather than being rounded confidently to one. Those two expected values +are then passed to WSSQ. + +What is searched, and what is not +--------------------------------- +Biomaterial concentrations, cell density and the printing parameters are +searched within user-supplied ranges. The CELL LINE IS NOT: it is chosen by the +user and held fixed for the run. This matters more than it looks. WSSQ falls +back to printability alone when Cell Response is 1, and an acellular +formulation therefore scores on printability only - so an optimiser allowed to +vary the cell line could raise its objective simply by removing the cells. +Fixing the line makes the acellular case a mode the user selects, not a +degenerate optimum the search can drift into. + +Extrapolation is reported, not prevented +---------------------------------------- +Nothing stops a user setting a range beyond anything in the corpus, and nothing +here silently clips it. `distance_report` instead states how far the returned +optimum sits outside the observed range of each variable and how close it is to +the nearest real formulation, so a candidate that the models are extrapolating +to is visible as such rather than presented with the same confidence as one +inside the training distribution. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field + +import numpy as np +import pandas as pd + +from mlate import config as cfg +from mlate import wssq as wssq_mod + +# Optuna's own logging is noisy inside a UI and uninformative in a script. +try: # pragma: no cover - import guard only + import optuna + optuna.logging.set_verbosity(optuna.logging.WARNING) +except Exception: # pragma: no cover + optuna = None + + +@dataclass +class Variable: + """One searchable input, with the range and grid the user specified.""" + name: str + low: float + high: float + step: float | None = None + + def suggest(self, trial): + if self.low >= self.high: + return float(self.low) + if self.step: + return float(trial.suggest_float(self.name, self.low, self.high, + step=self.step)) + return float(trial.suggest_float(self.name, self.low, self.high)) + + +@dataclass +class SearchSpace: + """Everything the optimiser may vary, plus the cell line it may not.""" + cell_line: str + biomaterials: list[Variable] = field(default_factory=list) + printing: list[Variable] = field(default_factory=list) + cell_density: Variable | None = None + + @property + def is_acellular(self) -> bool: + return self.cell_line == cfg.ACELLULAR_TOKEN + + +def class_distribution(estimator, X: np.ndarray, classes: np.ndarray + ) -> tuple[np.ndarray, list[dict[float, float]]]: + """ + Expected class values and the distributions they were taken over. + + Both come from a single `predict_proba` over the whole batch, because the + two are wanted together and a second call would double the cost of a + search. They carry different information and both are used. An expected + value of 2.5 is what a confident prediction split evenly between classes 2 + and 3 produces, and also what a diffuse prediction over 1 and 4 produces; + the optimiser needs only the number, while the protocol-generation prompt + is given the distribution so that it can state which case it describes. + + Falls back to the predicted label when a model exposes no probabilities; + five of the conventional classifiers do not, and for those the objective is + a step function rather than a smooth one and the distributions are empty. + """ + labels = np.asarray(classes, dtype=float) + if not hasattr(estimator, "predict_proba"): + idx = np.asarray(estimator.predict(X)).astype(int) + return labels[idx], [{} for _ in idx] + proba = np.asarray(estimator.predict_proba(X), dtype=float) + return proba @ labels, [{float(l): float(pr) for l, pr in zip(labels, row)} + for row in proba] + + +def expected_class_value(estimator, X: np.ndarray, + classes: np.ndarray) -> np.ndarray: + """ + Probability-weighted mean of the class labels. + + Falls back to the predicted label when a model exposes no probabilities; + five of the conventional classifiers do not, and for those the objective is + a step function rather than a smooth one. The application ranks models by + benchmarked performance and the probability-capable models occupy the top of + both leaderboards, so this path is a safeguard rather than the normal case. + """ + labels = np.asarray(classes, dtype=float) + if hasattr(estimator, "predict_proba"): + return np.asarray(estimator.predict_proba(X), dtype=float) @ labels + idx = np.asarray(estimator.predict(X)).astype(int) + return labels[idx] + + +# Candidates scored per pass through the models. Thirty-two because the cost +# of an in-context model is the pass over its context and is flat in the batch +# size up to at least sixty-four, while a larger batch buys the sampler fewer +# updates for no further saving. +DEFAULT_BATCH_SIZE = 32 + + +def _as_scorer(model): + """ + Accept either a pickled classifier bundle or a `mlate.serving.Servable`. + + The three model families no longer share a storage format - a deep network + is a state dictionary and a foundation model is a stored context - so the + application passes the uniform wrapper from `mlate.serving`. A plain bundle + is still accepted, because the analysis stages load their own artefacts + directly and have no reason to go through the serving layer. + """ + if isinstance(model, dict): + return {"estimator": model["estimator"], + "classes": np.asarray(model["classes"], dtype=float)} + return {"estimator": model, "classes": np.asarray(model.classes, + dtype=float)} + + +class Objective: + """ + WSSQ of one candidate formulation, as the optimiser sees it. + + Holds the preprocessor and both models so they are loaded once rather than + per trial, and records every evaluated candidate so the search can be + inspected afterwards instead of yielding only its arg-max. + """ + + def __init__(self, space: SearchSpace, preprocessor, feature_columns, + print_model, cell_model, + print_weight: float = wssq_mod.DEFAULT_PRINT_WEIGHT, + cell_weight: float = wssq_mod.DEFAULT_CELL_WEIGHT): + self.space = space + self.pre = preprocessor + self.feature_columns = list(feature_columns) + self.print_model = _as_scorer(print_model) + self.cell_model = _as_scorer(cell_model) + self.print_weight = print_weight + self.cell_weight = cell_weight + self.history: list[dict] = [] + + def candidate_frame(self, values: dict | list[dict]) -> pd.DataFrame: + """One row per candidate, in the raw feature layout the preprocessor + expects.""" + batch = [values] if isinstance(values, dict) else list(values) + rows = [] + for v in batch: + row = {c: 0.0 for c in self.feature_columns} + row.update({k: x for k, x in v.items() if k in row}) + row[cfg.CELL_COLS[0]] = self.space.cell_line + rows.append(row) + return pd.DataFrame(rows)[self.feature_columns] + + def evaluate_batch(self, batch: list[dict]) -> list[dict]: + """ + Score many candidates in one pass through each model. + + This is the form the search uses, and the reason a foundation model can + be offered at all. Their cost is dominated by the pass over the 2,646 + context rows, which is paid once per call however many candidates ride + in it: on this corpus a batch of thirty-two costs what a single row + costs. Scoring candidates one at a time, as the previous release did, + charged that pass once per candidate and made an in-context model look + two orders of magnitude more expensive than it is. The conventional + classifiers gain from it too, for the ordinary reason that per-call + Python overhead is then divided across the batch. + """ + X = np.asarray(self.pre.transform(self.candidate_frame(batch)), + dtype=float) + + exp_p, proba_p = class_distribution( + self.print_model["estimator"], X, self.print_model["classes"]) + + # An acellular formulation has no cell response to predict. Asking the + # model anyway would feed the optimiser a number with no referent; the + # rating scale defines class 1 as "not applicable", so that is what is + # used and WSSQ takes its printability-only branch. + if self.space.is_acellular: + exp_c = np.ones(len(batch)) + proba_c = [{}] * len(batch) + else: + exp_c, proba_c = class_distribution( + self.cell_model["estimator"], X, self.cell_model["classes"]) + + out = [] + for i in range(len(batch)): + out.append({ + "wssq": float(wssq_mod.compute_wssq( + float(exp_p[i]), float(exp_c[i]), + self.print_weight, self.cell_weight)), + "expected_printability": float(exp_p[i]), + "expected_cell_response": float(exp_c[i]), + "printability_proba": proba_p[i], + "cell_response_proba": proba_c[i]}) + return out + + def evaluate(self, values: dict) -> dict: + """Score one explicit formulation. Used to report the winner.""" + return self.evaluate_batch([values])[0] + + def suggest(self, trial) -> dict: + """One candidate drawn from the search space, as parameter values.""" + values = {v.name: v.suggest(trial) + for v in self.space.biomaterials + self.space.printing} + if self.space.cell_density is not None and not self.space.is_acellular: + values[cfg.CELL_COLS[1]] = self.space.cell_density.suggest(trial) + else: + values[cfg.CELL_COLS[1]] = 0.0 + return values + + def record(self, values: dict, out: dict) -> None: + """Append one evaluated candidate to the search history.""" + # history is exported to a spreadsheet; the two probability dictionaries + # do not belong in it. + self.history.append({**values, **{k: v for k, v in out.items() + if not k.endswith("_proba")}}) + + def __call__(self, trial) -> float: + """Sequential form, kept for a caller that drives Optuna directly.""" + values = self.suggest(trial) + out = self.evaluate(values) + self.record(values, out) + return out["wssq"] + + +def optimise(objective: Objective, n_trials: int = 100, + seed: int = cfg.RANDOM_STATE, progress=None, + batch_size: int = DEFAULT_BATCH_SIZE): + """ + Run the search and return (best_values, best_score, study). + + Tree-structured Parzen estimation, which is the Bayesian method used in the + published version: it models P(parameters | score) and samples where good + scores are likely, rather than sampling the space uniformly. `multivariate` + and `group` let it model interactions between parameters instead of treating + each in isolation, which matters here because printability depends on + combinations - a concentration that prints well at one nozzle diameter does + not at another. + + Candidates are drawn and scored in batches through Optuna's ask-and-tell + interface rather than one at a time. The trade is small and one-sided: the + candidates within a batch are proposed from the same posterior, so the + sampler sees `batch_size` fewer updates over a run, against a reduction in + scoring cost of one to two orders of magnitude. It is what makes the + in-context foundation models usable in a search at all - see + `Objective.evaluate_batch` - and it leaves the conventional classifiers + faster as well. Set `batch_size=1` to recover strictly sequential + behaviour. + """ + if optuna is None: # pragma: no cover + raise RuntimeError("optuna is required for scaffold optimisation") + + sampler = optuna.samplers.TPESampler( + seed=seed, n_startup_trials=min(30, max(5, n_trials // 4)), + multivariate=True, group=True, consider_prior=True) + study = optuna.create_study(direction="maximize", sampler=sampler) + + done = 0 + while done < n_trials: + size = min(max(1, batch_size), n_trials - done) + trials = [study.ask() for _ in range(size)] + batch = [objective.suggest(t) for t in trials] + results = objective.evaluate_batch(batch) + for trial, values, out in zip(trials, batch, results): + objective.record(values, out) + study.tell(trial, out["wssq"]) + done += size + if progress is not None: + progress(done, n_trials, study.best_value) + + return study.best_trial.params, float(study.best_value), study + + +def distance_report(values: dict, corpus: pd.DataFrame) -> dict: + """ + How far outside the observed data the proposed formulation sits. + + Two numbers, because they answer different questions. `out_of_range` names + the variables whose value exceeds anything recorded in the corpus - a + concentration no one has published. `nearest_neighbour_distance` is the + scaled Euclidean distance to the closest real formulation, which catches the + subtler case where every individual value is ordinary but the combination + has never been attempted. + """ + out_of_range = {} + for name, v in values.items(): + if name not in corpus.columns: + continue + col = pd.to_numeric(corpus[name], errors="coerce") + col = col[col > 0] + if len(col) == 0: + continue + lo, hi = float(col.min()), float(col.max()) + if v > 0 and (v < lo or v > hi): + out_of_range[name] = {"value": float(v), "observed_min": lo, + "observed_max": hi} + + shared = [c for c in values if c in corpus.columns] + nn = float("nan") + if shared: + sub = corpus[shared].apply(pd.to_numeric, errors="coerce").fillna(0.0) + span = (sub.max() - sub.min()).replace(0, 1.0) + target = pd.Series({c: values[c] for c in shared}) + nn = float((((sub - target) / span) ** 2).sum(axis=1).pow(0.5).min()) + + return {"n_out_of_range": len(out_of_range), + "out_of_range": out_of_range, + "nearest_neighbour_distance": nn} diff --git a/mlate/panels.py b/mlate/panels.py new file mode 100644 index 0000000000000000000000000000000000000000..3ffc75fcd2083cb754ccb62fe797bc06a791c80d --- /dev/null +++ b/mlate/panels.py @@ -0,0 +1,260 @@ +""" +Composite benchmark panels +========================== + +One reusable figure builder, used by the conventional-ML benchmark and by the +deep-learning/foundation benchmark, so the two figures share a visual grammar +and - critically - a metric order. + +The layout answers three questions in one read: + + A which model is best, and what does study grouping cost it + B how does it score on every other metric, random split + C does that profile survive grouping + +All panels share one y-axis: the models, sorted once by their random-split F1. +That shared order is what lets the eye travel horizontally from "this model +wins" to "and here is its whole metric profile under both protocols". + +Encodings, all from style.py +---------------------------- +protocol navy = random, slate = study-grouped, on the dumbbell dots +family a colour rug down the left edge of panel A, keyed to the same + eleven-family palette Figures S9-S11 already use. Family cannot go + on the dots, because the dots carry protocol; a rug adds the second + categorical dimension without spending a third visual channel. +metrics order comes from style.metric_columns() and is never passed in - a + per-figure metric list is what produced the inconsistency Referee 1 + flagged in comment 5. + +Sorting on the random split is deliberate: the navy dots then descend +monotonically while the slate dots scatter, and that scatter is the finding, +because the two protocols do not agree on which model is best. +""" + +from __future__ import annotations + +import numpy as np +import pandas as pd +from matplotlib.lines import Line2D +from matplotlib.patches import Rectangle + +from mlate import style as ms + +HEADLINE = "weighted_f1" + + +def _heat(ax, frame: pd.DataFrame, cols: list[str], annotate: bool, + tag: str, title: str): + """One metric heatmap, rows already in the plotted order.""" + data = frame[cols].to_numpy(dtype=float) + # Value text shrinks with the row count so every cell stays labelled even + # at thirty-one rows. Below ~4 pt the glyphs stop being legible in print, + # so that is the floor; the panel gets taller rather than the text smaller. + n_rows = data.shape[0] + vsize = 5.2 if n_rows <= 12 else (4.6 if n_rows <= 22 else 4.2) + # origin="lower" is mandatory, not cosmetic. imshow defaults to row 0 at + # the TOP while a scatter puts y=0 at the BOTTOM, so the default silently + # flips this panel relative to A and pairs every model with another + # model's metric profile. + ax.imshow(np.clip(data, 0.0, 1.0), cmap=ms.SEQ_NAVY, vmin=0.0, vmax=1.0, + aspect="auto", origin="lower", interpolation="nearest") + + for i in range(data.shape[0]): + for j in range(data.shape[1]): + v = data[i, j] + if np.isnan(v): + # Not missing data: these estimators expose no predict_proba, + # so ROC-AUC is undefined rather than unmeasured. Marked + # rather than left blank, so a reader does not read an empty + # cell as a zero. + ax.add_patch(Rectangle((j - .5, i - .5), 1, 1, + facecolor=ms.MISSING, lw=0)) + if annotate: + ax.text(j, i, "n/a", ha="center", va="center", + size=vsize - 0.3, color=ms.MUTED) + continue + # A sequential ramp cannot express a negative MCC or kappa, and + # "worse than chance" is a qualitative statement rather than a + # paler shade of the same thing. + if v < 0: + ax.add_patch(Rectangle((j - .5, i - .5), 1, 1, + facecolor=ms.RUST, lw=0)) + if annotate: + # Leading zero kept: ".79" saves one glyph and costs the + # reader a decision about what the value is. Bold because the + # numbers sit on a mid-tone ramp where regular weight at + # 4-5 pt loses contrast against the darker cells. + ax.text(j, i, f"{v:.2f}", ha="center", va="center", + size=vsize, weight="bold", + color="white" if (v > .62 or v < 0) else ms.TEXT) + + ax.set_xticks(range(len(cols))) + ax.set_xticklabels([ms.METRIC_LABELS[c] for c in cols], rotation=90, + size=6) + ax.set_yticks([]) + ax.set_title(title, size=6.6, color=ms.MUTED, pad=5) + ax.text(-0.02, 1.0, tag, transform=ax.transAxes, size=8.5, + weight="bold", ha="right", va="bottom", color=ms.TEXT) + ax.grid(False) + for side in ("top", "right", "left", "bottom"): + ax.spines[side].set_visible(False) + ax.tick_params(length=0) + + +def benchmark_panel(board: pd.DataFrame, *, families: pd.Series | None = None, + baselines: dict | None = None, + annotate_heat: bool = True, + sort_protocol: str = "random", + row_height: float = 0.155, + title: str = "", subtitle: str = ""): + """ + Three-panel benchmark figure for one target. + + `board` carries one row per (model, protocol) for the TEST partition, + already filtered to a single selection, with metric columns named as in + style.METRIC_ORDER. `families` maps model name to family for the colour + rug; omit it and the rug is dropped. + """ + cols = [c for c in ms.metric_columns() if c in board.columns] + wide = board.pivot_table(index="model", columns="protocol", + values=HEADLINE) + if sort_protocol not in wide.columns: + sort_protocol = wide.columns[0] + order = wide.sort_values(sort_protocol, ascending=True).index.tolist() + n = len(order) + + fam_present = [] + if families is not None: + fam_present = [f for f in ms.FAMILY_COLORS + if f in set(families.reindex(order).dropna())] + + legend_rows = 1 + (len(fam_present) > 0) + fig_h = max(2.6, n * row_height + 1.25 + 0.28 * legend_rows) + fig = ms.plt.figure(figsize=(ms.WIDTHS["double"], fig_h)) + + bottom = (0.34 + 0.20 * legend_rows) / fig_h + top = 1 - (0.78 if title else 0.34) / fig_h + gs = fig.add_gridspec(1, 3, width_ratios=[2.25, 1.42, 1.42], + wspace=0.07, left=0.235, right=0.985, + top=top, bottom=bottom) + + ax = fig.add_subplot(gs[0, 0]) + y = np.arange(n) + rnd = wide.reindex(order).get("random", pd.Series(np.nan, index=order)) + doi = wide.reindex(order).get("doi", pd.Series(np.nan, index=order)) + rnd, doi = rnd.to_numpy(dtype=float), doi.to_numpy(dtype=float) + + for i in range(n): + if np.isfinite(rnd[i]) and np.isfinite(doi[i]): + ax.plot([doi[i], rnd[i]], [i, i], color=ms.GRIDLINE, lw=1.0, + zorder=1, solid_capstyle="round") + ax.scatter(doi, y, s=21, color=ms.PROTOCOL_COLORS["doi"], zorder=3, + linewidths=0) + ax.scatter(rnd, y, s=21, color=ms.PROTOCOL_COLORS["random"], zorder=4, + linewidths=0) + + # Each value is placed on the OUTWARD side of its own dot - away from the + # other protocol's dot - rather than always right/always left. When the two + # scores are close the fixed-side version drew both labels into the gap + # between them, where they overlapped each other and the markers. Outward + # placement cannot collide however near the dots are, and colour still + # says which label belongs to which protocol. + for i in range(n): + a, b = rnd[i], doi[i] + rnd_right = (not np.isfinite(b)) or a >= b + if np.isfinite(a): + dx, ha = (4.5, "left") if rnd_right else (-4.5, "right") + ax.annotate(f"{a:.3f}", (a, i), xytext=(dx, 0), + textcoords="offset points", va="center", ha=ha, + size=5.1, color=ms.PROTOCOL_COLORS["random"], + path_effects=ms.halo(1.8), zorder=6) + if np.isfinite(b): + dx, ha = (-4.5, "right") if rnd_right else (4.5, "left") + ax.annotate(f"{b:.3f}", (b, i), xytext=(dx, 0), + textcoords="offset points", va="center", ha=ha, + size=5.1, color=ms.PROTOCOL_COLORS["doi"], + path_effects=ms.halo(1.8), zorder=6) + + # One baseline per protocol, not one for the figure: the two protocols + # have different test partitions with different class balance, so the + # majority-class score differs between them (printability: 0.333 random + # vs 0.375 grouped). A single line would misplace one of the two. + for proto, value in (baselines or {}).items(): + ax.axvline(value, color=ms.PROTOCOL_COLORS.get(proto, ms.RUST), + lw=0.9, ls=(0, (3, 2)), zorder=2, alpha=0.75) + + ax.set_yticks(y) + ax.set_yticklabels([ms.short(m) for m in order], size=6) + ax.set_ylim(-0.7, n - 0.3) + finite = np.concatenate([rnd[np.isfinite(rnd)], doi[np.isfinite(doi)]]) + if finite.size: + lo, hi = float(finite.min()), float(finite.max()) + pad = max(0.05, (hi - lo) * 0.17) + ax.set_xlim(lo - pad, hi + pad) + ax.set_xlabel("weighted F1", size=7) + ms.grid_axis(ax, "x") + ms.despine(ax, keep=("bottom",)) + # pad clears the family rug, which occupies the strip just left of the + # axes; without it the labels are drawn underneath the rug and clipped. + ax.tick_params(axis="y", length=0, pad=11 if families is not None + else 2) + ax.set_title("weighted F1 by validation protocol", size=6.6, + color=ms.MUTED, pad=5) + ax.text(-0.315, 1.0, "A", transform=ax.transAxes, size=8.5, + weight="bold", ha="left", va="bottom", color=ms.TEXT) + + # Family rug: a slim band between the labels and the plot, so family is + # readable without spending the dot colour, which encodes protocol. + if families is not None: + fam = families.reindex(order) + rug = ax.inset_axes([-0.026, 0.0, 0.017, 1.0], + transform=ax.transAxes) + for i, f in enumerate(fam): + rug.add_patch(Rectangle( + (0, i - 0.5), 1, 1, + facecolor=ms.FAMILY_COLORS.get(f, ms.SLATE), lw=0)) + rug.set_xlim(0, 1) + rug.set_ylim(-0.7, n - 0.3) + rug.set_xticks([]) + rug.set_yticks([]) + rug.grid(False) + for side in ("top", "right", "left", "bottom"): + rug.spines[side].set_visible(False) + + for k, proto in enumerate(("random", "doi")): + if proto not in set(board["protocol"]): + continue + axh = fig.add_subplot(gs[0, 1 + k]) + block = board[board["protocol"] == proto].set_index("model") + _heat(axh, block.reindex(order), cols, annotate_heat, "BC"[k], + ms.PROTOCOL_LABELS[proto]) + + handles = [Line2D([], [], marker="o", ls="none", markersize=4.2, + markerfacecolor=ms.PROTOCOL_COLORS[p], + markeredgecolor="none", label=ms.PROTOCOL_LABELS[p]) + for p in ("random", "doi") if p in set(board["protocol"])] + if baselines: + handles.append(Line2D([], [], color=ms.MUTED, lw=0.9, ls=(0, (3, 2)), + label="majority-class baseline (per protocol)")) + fig.legend(handles=handles, loc="lower center", + bbox_to_anchor=(0.5, (0.06 + 0.20 * (legend_rows - 1)) / fig_h), + ncol=len(handles), frameon=False, fontsize=6.4, + handletextpad=0.4, columnspacing=1.6) + + if fam_present: + fh = [Line2D([], [], marker="s", ls="none", markersize=4.4, + markerfacecolor=ms.FAMILY_COLORS[f], + markeredgecolor="none", + label=ms.FAMILY_LABELS.get(f, f)) for f in fam_present] + fig.legend(handles=fh, loc="lower center", + bbox_to_anchor=(0.5, 0.02 / fig_h), + ncol=min(7, len(fh)), frameon=False, fontsize=6.0, + handletextpad=0.35, columnspacing=1.1) + + if title: + fig.suptitle(title, size=9.5, y=1 - 0.14 / fig_h, va="top") + if subtitle: + fig.text(0.5, 1 - 0.36 / fig_h, subtitle, ha="center", va="top", + size=6.4, color=ms.MUTED) + return fig diff --git a/mlate/pipeline.py b/mlate/pipeline.py new file mode 100644 index 0000000000000000000000000000000000000000..ac2753e4b338b78e7c85878578b3436d0940930c --- /dev/null +++ b/mlate/pipeline.py @@ -0,0 +1,118 @@ +""" +The preprocessing pipeline +========================== + +Builds an UNFITTED sklearn transformer. Callers fit it on a fold's training +rows and apply it to that fold's test rows, so nothing about the test set can +reach the model - not a scaler range, not an imputed value, not an encoding. + +Layout, matching the published methods: + + Cell Line BinaryEncoder high-cardinality categorical, + ~8 columns instead of one-hot's + 187, and no false ordinal order + Cell Density median by line then MinMax + Biomaterials present-median then MinMax + Ambient temperatures constant + flag then MinMax + Other print parameters model-based + flag then MinMax + +MinMax is applied last and fitted on training rows only, matching the scaling +used in the submitted version. +""" + +from __future__ import annotations + +import numpy as np +import pandas as pd +from category_encoders.binary import BinaryEncoder +from sklearn.compose import ColumnTransformer +from sklearn.pipeline import Pipeline +from sklearn.preprocessing import MinMaxScaler + +from mlate import config as cfg +from mlate.dataset import Columns +from mlate.imputation import ( + AmbientFill, CellDensityFill, MedianFill, ModelBasedFill, + PresentMedianFill, +) + + +def build_preprocessor(columns: Columns, + scale: bool = True, + model_based: bool = True) -> ColumnTransformer: + """ + Return an unfitted preprocessor. + + scale MinMax the numeric blocks. Turn off for tree models that do + not need it and for interpretability work. + model_based Use iterative imputation for the unknown-if-missing printing + parameters. Turn off for a fast median baseline. + """ + def numeric(step): + return Pipeline([("impute", step), + ("scale", MinMaxScaler() if scale else "passthrough")]) + + unknown = [c for c in cfg.UNKNOWN_IF_MISSING if c in columns.print_params] + ambient = [c for c in cfg.AMBIENT_COLUMNS if c in columns.print_params] + + # The masking test favours model-based filling for the ambient columns too, + # so by default they join the same block. Both routes keep the [reported] + # indicator, which is what carries the informative-missingness signal. + if cfg.AMBIENT_STRATEGY == "model_based": + unknown, ambient = unknown + ambient, [] + elif cfg.AMBIENT_STRATEGY != "constant": + raise ValueError(f"unknown AMBIENT_STRATEGY: {cfg.AMBIENT_STRATEGY}") + + transformers = [ + ("cell_line", BinaryEncoder( + cols=[columns.cell_line], return_df=False, + handle_unknown="value", handle_missing="value"), + [columns.cell_line]), + ("cell_density", numeric(CellDensityFill()), + [columns.cell_line, columns.cell_density]), + ("biomaterials", numeric(PresentMedianFill()), columns.biomaterials), + ("printing", numeric( + ModelBasedFill() if (model_based and + cfg.PRINTING_IMPUTATION == "model_based") + else MedianFill()), unknown), + ] + if ambient: + transformers.insert(3, ("ambient", numeric(AmbientFill()), ambient)) + + return ColumnTransformer( + transformers=transformers, + remainder="drop", + verbose_feature_names_out=False, + n_jobs=None, + ) + + + +def feature_names(preprocessor: ColumnTransformer) -> list[str]: + """Output feature names of a fitted preprocessor.""" + try: + return [str(n) for n in preprocessor.get_feature_names_out()] + except Exception: + n = sum( + t.transform(np.zeros((1, len(c)))).shape[1] if hasattr(t, "transform") + else len(c) + for _, t, c in preprocessor.transformers_ if c + ) + return [f"f{i}" for i in range(n)] + + +def fit_transform_fold(df: pd.DataFrame, columns: Columns, + train_idx, test_idx, **kw + ) -> tuple[np.ndarray, np.ndarray, ColumnTransformer]: + """ + Fit on the fold's training rows, transform both sides. + + This is the only sanctioned way to preprocess for an evaluation: fitting + once on the whole frame and slicing afterwards leaks scaler ranges and + imputed values from test into train. + """ + pre = build_preprocessor(columns, **kw) + X = df[columns.predictors] + X_train = pre.fit_transform(X.iloc[train_idx]) + X_test = pre.transform(X.iloc[test_idx]) + return np.asarray(X_train, float), np.asarray(X_test, float), pre diff --git a/mlate/prompts/protocol_v2.md b/mlate/prompts/protocol_v2.md new file mode 100644 index 0000000000000000000000000000000000000000..5de76375731d244317bf997020f761a608535107 --- /dev/null +++ b/mlate/prompts/protocol_v2.md @@ -0,0 +1,137 @@ +# MLATE protocol-generation prompt, version 2 + +Versioned as a file rather than embedded in the application, so that the exact +instruction behind any generated protocol can be cited, diffed and reproduced. +Version 1 is the prompt shipped with the previous release. + +Everything above the first section marker is discarded by +`mlate.protocol.build_prompt` and never reaches the model, which is why these +notes are here and not at the end of the file. The marker itself is deliberately +not written out in this paragraph: the parser splits on its first occurrence, +so quoting it here would make the prompt begin in the middle of a comment. +Placeholders in `{braces}` are filled in by the same function. + +Changes from version 1, all directed at reliability rather than style: + +1. The nearest published formulations from the corpus are supplied as + precedent, so generation is grounded in recorded practice rather than in + prior expectation. +2. Predicted outcomes arrive with their class probabilities and are labelled as + predictions, so a hedged prediction and a confident one can be told apart. +3. A refusal path: implausible, missing or substituted values are reported in + *Deviations and cautions* rather than filled in silently. +4. Supplier names, catalogue numbers and citations are prohibited outright. +5. Reasoning, planning and self-verification are excluded from the output. + Added after two models under test returned their own rule-compliance + checklists as part of the document. +6. Where a parameter lies outside the corpus, the observed bounds are supplied + with it. Added after a model, given only the parameter name, stated a corpus + range it had invented. + +--- + +## SYSTEM + +You are a senior tissue-engineering experimentalist writing a bench protocol for +another experienced experimentalist. You write only what you can justify from +the parameters you are given and from standard, widely used laboratory practice. + +Absolute rules, which override every other instruction: + +1. **Never invent specifics you were not given.** Do not state supplier names, + catalogue numbers, lot numbers, product codes, or literature citations. If a + reagent grade or a piece of equipment matters, describe it generically + ("a 25 G blunt-tip stainless-steel nozzle", "cell-culture-grade CaCl2"). +2. **Never present a predicted value as a measured one.** The printability and + cell-response figures you are given are model predictions with stated + uncertainty, not experimental results. +3. **Do not silently correct an implausible input.** If a supplied parameter is + physically implausible, or unsafe for the stated cell line, keep the supplied + value visible, state the problem, give the value you would use instead, and + put it in the *Deviations and cautions* section. Never quietly substitute. +4. **Say when you do not know.** If the parameters are insufficient to specify a + step, write what is missing rather than filling the gap with a plausible + number. +5. **No preamble, no summary, no closing remarks, no disclaimers** beyond the + *Deviations and cautions* section. Begin at heading 1. +6. **Output the protocol only.** Do not include planning, deliberation, + self-checks, or any verification of your own compliance with these rules. + Reasoning belongs in your reasoning, not in the document. The reader is at a + bench and every line that is not protocol is a line they must first identify + as not protocol. + +Write in the imperative, in numbered steps. Every quantity must carry a unit and +be measurable at the bench: volumes in mL, concentrations in % w/v or mg/mL, +times in min, temperatures in Β°C, speeds in mm/s, pressures in kPa. + +--- + +## USER + +A formulation has been proposed by an optimisation routine and now requires a +fabrication protocol. + +### Formulation + +Mode: {mode} +Cell line: {cell_line} +{cell_density_line} + +Biomaterials and concentrations: +{biomaterials} + +Printing and crosslinking parameters: +{printing} + +### Model predictions for this formulation + +These are predictions from models trained on {n_records} scaffold records +extracted from the literature. They are not measurements. + +{predictions} + +{confidence_note} + +### Precedent from the corpus + +The following real published formulations are the closest matches to the +proposed one in the training corpus. Use them to sanity-check plausibility. Do +not cite them, and do not copy values from them that contradict the proposed +formulation. + +{neighbours} + +### Extrapolation status + +{extrapolation} + +### What to produce + +Exactly these sections, in this order, with no others: + +1. Required Materials and Equipment +2. Sterilisation and Safety Precautions +3. {step3} +4. Printing Settings and Execution +5. Post-processing and Crosslinking +6. {step6} +7. Quality-control Checkpoints +8. Deviations and Cautions + +Requirements for the content: + +- Give exact timings, temperatures and workflow order. +- Address the failure modes that the supplied parameters actually make likely + (for example nozzle clogging at small diameters with high-viscosity inks, + shear-induced loss of viability at high extrusion pressure, premature + gelation at elevated syringe temperature, filament fusion at low nozzle + movement speed). Address only those that apply, and say why each applies. +- Give quality-control checkpoints that can be performed with standard + equipment, with the criterion for pass or fail stated as a number wherever + one exists. +- In *Deviations and Cautions*, list every value you changed, every value you + believe is implausible, everything the parameters left unspecified, and any + respect in which this formulation lies outside the region the models were + trained on. + +{user_constraints} diff --git a/mlate/protocol.py b/mlate/protocol.py new file mode 100644 index 0000000000000000000000000000000000000000..21eb5eb1201d1fb9f9e0f94feffd286e4d17a98e --- /dev/null +++ b/mlate/protocol.py @@ -0,0 +1,611 @@ +""" +LLM-assisted protocol generation +================================ + +Turns an optimised formulation into a bench-ready fabrication protocol. + +Provider +-------- +Requests go to OpenRouter, which exposes many vendors' models behind one +OpenAI-compatible endpoint. The previous release called Google Gemini directly, +which tied the module to one vendor's SDK, one account and one model family. A +single endpoint means a user can reach Anthropic, OpenAI, Google, Meta and open +models with one key, and it means this module does not have to be rewritten +when a vendor changes its client library. The request body is plain JSON over +HTTPS, so there is no provider SDK to depend on at all. + +What makes the output trustworthy, or not +----------------------------------------- +A language model asked for a laboratory protocol will produce one whether or not +it has grounds to. Four things here are aimed at that, and all of them are in +the prompt rather than in post-processing, because a fabricated catalogue number +cannot be detected after the fact: + + grounding the closest real formulations from the corpus are supplied, so + the model has published precedent in front of it rather than + only its own priors + uncertainty predictions are passed with their class probabilities and + labelled as predictions, so the protocol cannot present a + modelled value as a measured one + refusal path the model is told to report what is missing or implausible + instead of filling the gap, and is given a section to put it in + no invention supplier names, catalogue numbers and citations are forbidden + outright, since those are what a model confabulates most + readily and what a reader is least able to check + +Determinism and record-keeping +------------------------------ +Temperature and top-p are low and explicit, and every generation returns the +model id, the parameters, the prompt version and a hash of the exact prompt +sent. A protocol that cannot be traced to the instruction that produced it is +not reproducible, and the referee asked specifically for the model and the +generation parameters to be stated. + +The API key is read from the environment or a git-ignored .env file, or passed +in by the caller. It is never written to a source file and never logged. +""" + +from __future__ import annotations + +import hashlib +import json +import os +import time +import urllib.error +import urllib.request +from dataclasses import dataclass, field +from datetime import datetime, timezone +from pathlib import Path + +import numpy as np +import pandas as pd + +from mlate import config as cfg + +ENDPOINT = "https://openrouter.ai/api/v1/chat/completions" +PROMPT_DIR = Path(__file__).resolve().parent / "prompts" +PROMPT_VERSION = "protocol_v2" + +DEFAULT_TEMPERATURE = 0.15 +DEFAULT_TOP_P = 0.85 +# 6144 was too tight and truncated the output of every verbose model tested, +# losing the Deviations and Cautions section - the one section whose absence is +# least visible and most consequential, since a protocol that simply stops +# looks complete. Every model in the curated list has a context window of at +# least 128k, so the ceiling costs nothing when it is not reached. +DEFAULT_MAX_TOKENS = 16384 + +# A short, curated list rather than OpenRouter's full catalogue of several +# hundred. These span vendors and price points so that a user without a paid +# account can still generate a protocol. +# +# The catalogue turns over quickly - five of the eleven identifiers written here +# for the previous revision had been withdrawn within weeks, and a withdrawn +# identifier fails at generation time with a message about the model not being +# available. `available_models` therefore checks this list against the live +# catalogue before it is shown, and `fetch_models` returns the catalogue itself +# for a user wanting something not listed. The list is a starting point, not the +# authority. +# +# Tiers: "free" costs nothing and is aggressively rate-limited, so a request may +# be refused and should simply be retried or another model chosen; "cheap" is +# under about $1 per million output tokens; "paid" is the strongest available. +SUGGESTED_MODELS = [ + # Free. `openrouter/free` is OpenRouter's own routed endpoint rather than a + # single model, which is why it heads the list and is the default: a named + # free model is shared and frequently refuses under load, and the routed + # endpoint moves to whichever is answering. + ("openrouter/free", "OpenRouter", "free", + "Free, auto-routed across providers. The most reliable free option and " + "the default, because a named free model is often busy. It selects the " + "model per request, so pick a named one for any protocol reported in a " + "paper; the generation record states which model actually served each " + "request either way."), + ("nvidia/nemotron-3-super-120b-a12b:free", "NVIDIA", "free", + "Free. Complete and well-formed in testing; caught a physically " + "implausible input rather than silently correcting it."), + ("nvidia/nemotron-3-ultra-550b-a55b:free", "NVIDIA", "free", + "Free, larger, slower. Complete in testing."), + ("minimax/minimax-m3:free", "MiniMax", "free", + "Free, very long context. Complete in testing."), + ("z-ai/glm-5.2:free", "Z.ai", "free", + "Free. Often rate-limited; retry or choose another."), + ("google/gemma-4-31b-it:free", "Google", "free", + "Free, smaller. Often rate-limited."), + + # Under roughly $1 per million output tokens. + ("qwen/qwen3.7-flash", "Alibaba", "cheap", + "Inexpensive and thorough; stated the cell-line mismatch between the " + "candidate and its nearest precedents unprompted."), + ("deepseek/deepseek-v3.2", "DeepSeek", "cheap", + "Inexpensive; read the class probabilities correctly in testing."), + ("z-ai/glm-5.3-flash", "Z.ai", "cheap", + "Very cheap, long context, slow to respond."), + ("meta-llama/llama-4-maverick", "Meta", "cheap", + "Open-weight; useful if provider choice is constrained."), + ("mistralai/mistral-medium-3-5", "Mistral", "cheap", + "European provider, if data residency matters."), + + # Strongest available. These reserve the full token budget against the + # account balance before generating, so a nearly empty account is refused + # here while the cheap models still run. + ("anthropic/claude-opus-4.5", "Anthropic", "paid", + "Strongest reasoning; best adherence to the no-invention rules."), + ("anthropic/claude-sonnet-5", "Anthropic", "paid", + "Near-Opus quality, faster and cheaper."), + ("openai/gpt-5.2", "OpenAI", "paid", + "Strong general instruction-following."), + ("google/gemini-3.1-pro-preview", "Google", "paid", + "Long context; the closest equivalent to the previous release."), + ("x-ai/grok-4.3", "xAI", "paid", + "Long context, strong technical writing."), +] + +# A free model, so the feature works on a new account with no credit. The +# paid models reserve `max_tokens` against the balance before generating and +# refuse outright when it will not cover them, which is what a user with an +# empty account meets first. +DEFAULT_MODEL = "openrouter/free" + + +def api_key(explicit: str | None = None) -> str | None: + """Key from the caller, the environment, or a git-ignored .env file.""" + if explicit: + return explicit.strip() + for var in ("OPENROUTER_API_KEY", "OPENROUTER_KEY"): + if os.environ.get(var): + return os.environ[var].strip() + for candidate in (cfg.ROOT / ".env", Path.cwd() / ".env"): + if candidate.exists(): + for line in candidate.read_text(encoding="utf-8").splitlines(): + if line.strip().startswith("OPENROUTER_API_KEY"): + return line.split("=", 1)[1].strip().strip('"').strip("'") + return None + + +def key_status(key: str | None) -> str: + """Describe a key without printing it.""" + if not key: + return "not set" + return f"set ({len(key)} chars, ending {key[-4:]})" + + +# ── prompt assembly ────────────────────────────────────────────────────────── + +@dataclass +class Formulation: + """Everything the prompt needs about one optimised candidate.""" + cell_line: str + biomaterials: dict[str, float] = field(default_factory=dict) + printing: dict[str, float] = field(default_factory=dict) + cell_density: float | None = None + expected_printability: float | None = None + expected_cell_response: float | None = None + printability_proba: dict | None = None + cell_response_proba: dict | None = None + wssq: float | None = None + neighbours: pd.DataFrame | None = None + extrapolation: dict | None = None + + @property + def is_acellular(self) -> bool: + return self.cell_line == cfg.ACELLULAR_TOKEN + + +def _fmt_proba(proba: dict | None) -> str: + if not proba: + return "" + parts = [f"class {k}: {100 * v:.0f}%" for k, v in sorted(proba.items()) + if v >= 0.01] + return ", ".join(parts) + + +def _neighbour_block(nb: pd.DataFrame | None) -> str: + if nb is None or len(nb) == 0: + return "No sufficiently similar formulation was found in the corpus." + # to_dict, not itertuples: the column names carry units and per-cent signs + # ("Alginate (%w/v)"), which are not valid Python identifiers, so + # itertuples silently renames them to _0, _1, _2 and the precedent reaches + # the model as a list of anonymous numbers. + lines = [] + for i, row in enumerate(nb.to_dict("records"), 1): + bits = [f"{k}: {v:g}" if isinstance(v, (int, float)) else f"{k}: {v}" + for k, v in row.items() + if pd.notna(v) and str(v) not in ("0", "0.0", "")] + lines.append(f"{i}. " + "; ".join(bits)) + return "\n".join(lines) + + +def build_prompt(f: Formulation, n_records: int = 2646, + user_constraints: str = "") -> tuple[str, str]: + """Return (system, user) messages from the versioned template.""" + template = (PROMPT_DIR / f"{PROMPT_VERSION}.md").read_text(encoding="utf-8") + system = template.split("## SYSTEM", 1)[1].split("## USER", 1)[0] + system = system.strip().lstrip("-").strip() + user_tpl = template.split("## USER", 1)[1].strip() + + bio = "\n".join(f"- {k}: {v:.3g}" for k, v in sorted(f.biomaterials.items()) + if v and v > 0) or "- none specified" + pp = "\n".join(f"- {k}: {v:.4g}" for k, v in sorted(f.printing.items()) + if v is not None) or "- none specified" + + preds = [] + if f.expected_printability is not None: + preds.append(f"- Predicted printability (scale 0-3, higher is better): " + f"{f.expected_printability:.2f}") + if f.printability_proba: + preds.append(f" class probabilities: " + f"{_fmt_proba(f.printability_proba)}") + if not f.is_acellular and f.expected_cell_response is not None: + preds.append(f"- Predicted cell response (scale 1-5, higher is better): " + f"{f.expected_cell_response:.2f}") + if f.cell_response_proba: + preds.append(f" class probabilities: " + f"{_fmt_proba(f.cell_response_proba)}") + if f.wssq is not None: + preds.append(f"- Combined scaffold-quality score (WSSQ): {f.wssq:.1f}%") + + # A confident prediction and a hedged one warrant different protocols; the + # model is told which it is looking at rather than left to assume. Both + # targets are checked, because they are independent: printability is + # routinely predicted at above 90% for one class while cell response is + # split across two, and a note that reported only the first would describe + # the formulation as well characterised when half of it is not. + hedged = [name for name, proba in + (("printability", f.printability_proba), + ("cell response", f.cell_response_proba)) + if proba and max(proba.values()) < 0.5] + if hedged: + which = " and ".join(hedged) + confidence = (f"The {which} prediction is not confident: no single " + f"class exceeds 50% probability. Treat the corresponding " + f"parameters as a starting point requiring pilot " + f"optimisation, and say so in the protocol.") + elif f.printability_proba or f.cell_response_proba: + confidence = ("The predictions above carry the stated class " + "probabilities. Where a probability is spread across " + "classes, treat the corresponding parameter as requiring " + "pilot verification.") + else: + # No distribution was supplied, so nothing is known about the spread. + # Claiming confidence here would be an assertion about a quantity that + # was never computed. + confidence = ("Class probabilities were not supplied for these " + "predictions, so their confidence is unknown. Do not " + "describe any predicted outcome as likely or reliable.") + + extrap = "The proposed formulation lies within the range of the corpus." + if f.extrapolation and f.extrapolation.get("n_out_of_range"): + # The observed bounds, not merely the names of the offending + # parameters. Given names alone a model states the range anyway and + # invents the numbers: one test run asserted an alginate range of + # "0.5-3 % w/v" for a corpus whose actual range is 0.25-20. An + # invented bound in a Deviations section is worse than no bound, + # because that section is where a reader looks to calibrate trust. + lines = [] + for name, d in list(f.extrapolation["out_of_range"].items())[:8]: + lines.append(f"- {name}: proposed {d['value']:.4g}, observed range " + f"in the corpus {d['observed_min']:.4g} to " + f"{d['observed_max']:.4g}") + listing = "\n".join(lines) + extrap = (f"{f.extrapolation['n_out_of_range']} parameter(s) lie " + f"outside the range observed in the corpus:\n\n{listing}\n\n" + f"The models are extrapolating for this candidate. State " + f"this explicitly in Deviations and Cautions, quoting these " + f"observed ranges and no others. Do not state a corpus range " + f"for any parameter not listed above: you have not been " + f"given one, and those parameters are within range.") + + user = user_tpl.format( + mode=("acellular 3D printing (no cells; cell response is not " + "applicable)" if f.is_acellular else "cell-laden bioprinting"), + cell_line=("none - acellular" if f.is_acellular else f.cell_line), + cell_density_line=("" if f.is_acellular else + f"Cell density: {f.cell_density:.3g} x10^6 cells/mL" + if f.cell_density else "Cell density: not specified"), + biomaterials=bio, + printing=pp, + n_records=f"{n_records:,}", + predictions="\n".join(preds) or "- none available", + confidence_note=confidence, + neighbours=_neighbour_block(f.neighbours), + extrapolation=extrap, + step3=("Ink Preparation" if f.is_acellular else "Bioink Preparation"), + step6=("Post-printing Handling and Storage" if f.is_acellular + else "Cell Culture and Incubation"), + user_constraints=( + f"### Additional constraints from the user\n\n{user_constraints}\n\n" + f"Integrate these into the protocol. If a constraint conflicts with " + f"the proposed parameters, follow the constraint and record the " + f"conflict in Deviations and Cautions." + if user_constraints.strip() else ""), + ) + return system, user + + +# ── generation ─────────────────────────────────────────────────────────────── + +def fetch_models(key: str | None = None, timeout: int = 20) -> list[dict]: + """Live OpenRouter catalogue, for users wanting a model not in the list.""" + req = urllib.request.Request("https://openrouter.ai/api/v1/models", + headers={"Accept": "application/json"}) + if key: + req.add_header("Authorization", f"Bearer {key}") + with urllib.request.urlopen(req, timeout=timeout) as r: + return json.loads(r.read().decode("utf-8")).get("data", []) + + +def available_models(key: str | None = None, + timeout: int = 20) -> tuple[list[tuple], bool]: + """ + The curated list, filtered to what the catalogue currently offers. + + Returns (models, verified). `verified` is False when the catalogue could + not be reached, in which case the full curated list is returned unfiltered + and the caller should say that availability is unconfirmed rather than + imply it was checked. + + This exists because a withdrawn identifier is indistinguishable from a + working one until a request is made, and the request then fails after the + user has already run an optimisation. Checking a single catalogue call in + advance moves that failure to a point where it costs nothing. + """ + try: + live = {m["id"] for m in fetch_models(key, timeout)} + except Exception: + return list(SUGGESTED_MODELS), False + if not live: + return list(SUGGESTED_MODELS), False + return [m for m in SUGGESTED_MODELS if m[0] in live], True + + +def _explain_http_error(e, model: str) -> str: + """ + Turn OpenRouter's response into something a user can act on. + + The raw body is a nested JSON envelope several hundred characters long, and + the three failures a user actually meets - a rate-limited free model, an + empty account, a withdrawn identifier - all read the same in it. + """ + try: + body = json.loads(e.read().decode("utf-8", errors="replace")) + detail = (body.get("error") or {}).get("message") or str(body) + raw = ((body.get("error") or {}).get("metadata") or {}).get("raw") + if raw: + detail = f"{detail} ({raw})" + except Exception: + detail = "no further detail" + + if e.code == 429: + return (f"{model} is rate-limited right now. Free models are shared " + f"and refuse requests under load; wait a minute and retry, or " + f"choose another model. ({detail})") + if e.code == 402: + return (f"{model} requires credit on the OpenRouter account and the " + f"balance is insufficient. Choose a model marked free, or add " + f"credit. ({detail})") + if e.code in (400, 404): + return (f"{model} was not accepted by OpenRouter - most often because " + f"the identifier has been withdrawn from the catalogue. Pick " + f"another model from the list. ({detail})") + if e.code in (401, 403): + return f"The OpenRouter API key was rejected. ({detail})" + return f"OpenRouter returned {e.code}: {detail}" + + +def _consume_stream(response, on_chunk) -> tuple[str, str, dict, str]: + """ + Read a server-sent-event stream and return (text, finish_reason, usage, + model). + + Chunks are handed to `on_chunk` as they arrive, so an interface can render + the protocol while it is being written. That matters more here than in most + applications: the free models a user without credit will reach take between + thirty seconds and two and a half minutes to finish, and a blank screen for + that long is indistinguishable from a hang. + + Lines beginning with a colon are stream comments used as keep-alives and + carry no data. + """ + parts, finish, usage, served_model = [], None, {}, None + for raw in response: + line = raw.decode("utf-8", errors="replace").strip() + if not line or line.startswith(":") or not line.startswith("data:"): + continue + body = line[5:].strip() + if body == "[DONE]": + break + try: + packet = json.loads(body) + except json.JSONDecodeError: + continue + + served_model = packet.get("model") or served_model + if packet.get("usage"): + usage = packet["usage"] + for choice in packet.get("choices") or []: + finish = choice.get("finish_reason") or finish + piece = (choice.get("delta") or {}).get("content") + if piece: + parts.append(piece) + if on_chunk is not None: + on_chunk(piece) + return "".join(parts), finish, usage, served_model + + +def _read_whole(response) -> tuple[str, str, dict, str]: + """The same tuple as `_consume_stream`, from one non-streamed response.""" + payload = json.loads(response.read().decode("utf-8")) + choice = (payload.get("choices") or [{}])[0] + message = choice.get("message") or {} + + # A reasoning model can return its answer under "reasoning" with "content" + # empty, and a truncated or refused completion returns content null as + # well. Reading the field blindly propagates None into the caller and fails + # somewhere unrelated, so the case is resolved here. + text = message.get("content") or message.get("reasoning") or "" + return (text, choice.get("finish_reason"), payload.get("usage") or {}, + payload.get("model")) + + +# Failures worth trying again rather than reporting. Free models are shared and +# refuse under load, and a provider occasionally closes a connection having +# sent nothing; both clear on their own within seconds. An insufficient balance +# or a withdrawn identifier never clears, and retrying those only delays the +# explanation. +RETRY_CODES = (429, 500, 502, 503, 504) +DEFAULT_RETRIES = 2 + + +def generate(formulation: Formulation, key: str | None = None, + model: str = DEFAULT_MODEL, + temperature: float = DEFAULT_TEMPERATURE, + top_p: float = DEFAULT_TOP_P, + max_tokens: int = DEFAULT_MAX_TOKENS, + user_constraints: str = "", n_records: int = 2646, + timeout: int = 300, on_chunk=None, + retries: int = DEFAULT_RETRIES, backoff: float = 4.0, + on_retry=None) -> dict: + """ + Generate a protocol and return it with everything needed to reproduce it. + + The returned record carries the model id, all generation parameters, the + prompt version and a SHA-256 of the exact prompt sent, so that a protocol + appearing in a paper or a supplement can be traced to the instruction that + produced it. + + Pass `on_chunk` to stream: it is called with each fragment as it arrives, + and the complete text is still returned in the record. `on_retry(attempt, + reason)` is called before each retry, for an interface that would rather + say what is happening than appear stalled. + """ + key = api_key(key) + if not key: + raise RuntimeError( + "No OpenRouter API key. Set OPENROUTER_API_KEY in the environment " + "or a .env file, or pass one in.") + + system, user = build_prompt(formulation, n_records, user_constraints) + digest = hashlib.sha256((system + "\n" + user).encode("utf-8")).hexdigest() + streaming = on_chunk is not None + + request_body = { + "model": model, + "messages": [{"role": "system", "content": system}, + {"role": "user", "content": user}], + "temperature": temperature, + "top_p": top_p, + "max_tokens": max_tokens, + # Let a reasoning model think, but return the protocol rather than the + # thinking. Without this the trace arrives inside the message content + # and is indistinguishable from the protocol to anyone reading it: one + # model under test returned its own rule-compliance checklist as part + # of the document. Providers that do not support the field ignore it. + "reasoning": {"exclude": True}, + } + if streaming: + request_body["stream"] = True + # Token counts arrive in a final packet rather than with each chunk. + request_body["stream_options"] = {"include_usage": True} + body = json.dumps(request_body).encode("utf-8") + + text = finish = served = None + usage: dict = {} + last_error = None + + for attempt in range(retries + 1): + if attempt: + if on_retry is not None: + on_retry(attempt, last_error) + time.sleep(backoff * attempt) + + req = urllib.request.Request( + ENDPOINT, data=body, + headers={"Authorization": f"Bearer {key}", + "Content-Type": "application/json", + "HTTP-Referer": "https://github.com/saeedrafieyan/mlate", + "X-Title": "MLATE V3"}) + emitted = False + + def watched(piece): + # Once a fragment has reached the caller's screen a retry would + # append a second protocol to the first, so the attempt is + # committed from the first chunk onward. + nonlocal emitted + emitted = True + on_chunk(piece) + + try: + with urllib.request.urlopen(req, timeout=timeout) as response: + if streaming: + text, finish, usage, served = _consume_stream( + response, watched) + else: + text, finish, usage, served = _read_whole(response) + except urllib.error.HTTPError as e: + last_error = _explain_http_error(e, model) + if e.code in RETRY_CODES and attempt < retries and not emitted: + continue + raise RuntimeError(last_error) from None + except (urllib.error.URLError, TimeoutError) as e: + last_error = f"Could not reach OpenRouter: {e}" + if attempt < retries and not emitted: + continue + raise RuntimeError(last_error) from None + + if text.strip(): + break + + # An empty completion with nothing reported as wrong. Seen + # intermittently on the free tier and it clears on a retry, so it is + # treated as a transient fault rather than shown to the user as one. + last_error = (f"{model} returned an empty completion " + f"(finish reason: {finish or 'none reported'})") + if attempt < retries and not emitted: + continue + raise RuntimeError( + f"{last_error}. If the reason is a length limit, raise the token " + f"budget; otherwise choose another model.") + + return { + "protocol": text, + "model": served or model, + "prompt_version": PROMPT_VERSION, + "prompt_sha256": digest, + "temperature": temperature, + "top_p": top_p, + "max_tokens": max_tokens, + "streamed": streaming, + "attempts": attempt + 1, + "truncated": finish == "length", + "prompt_tokens": usage.get("prompt_tokens"), + "completion_tokens": usage.get("completion_tokens"), + "generated_utc": datetime.now(timezone.utc).isoformat(timespec="seconds"), + } + + +def nearest_formulations(values: dict, corpus: pd.DataFrame, + columns, n: int = 3) -> pd.DataFrame: + """ + The closest real formulations in the corpus, for grounding the prompt. + + Distance is computed on the scaled numeric predictors so that a material at + 30 %w/v and a pressure at 300 kPa contribute comparably. Only the non-zero + components of each neighbour are shown, since a row listing 130 zeros is not + precedent a reader or a model can use. + """ + numeric = [c for c in columns.predictors + if c in corpus.columns + and pd.api.types.is_numeric_dtype(corpus[c])] + sub = corpus[numeric].apply(pd.to_numeric, errors="coerce").fillna(0.0) + span = (sub.max() - sub.min()).replace(0, 1.0) + target = pd.Series({c: float(values.get(c, 0.0)) for c in numeric}) + d = (((sub - target) / span) ** 2).sum(axis=1).pow(0.5) + + keep = d.nsmallest(n).index + show = corpus.loc[keep].copy() + cols = [c for c in (list(columns.biomaterials) + list(cfg.CELL_COLS) + + list(columns.print_params) + cfg.TARGETS) + if c in show.columns] + out = show[cols] + return out.loc[:, (out != 0).any(axis=0)] diff --git a/mlate/resources.py b/mlate/resources.py new file mode 100644 index 0000000000000000000000000000000000000000..d32bf33d2f320531e4b6bf6a60d1d5449e725778 --- /dev/null +++ b/mlate/resources.py @@ -0,0 +1,151 @@ +""" +Compute budget +============== + +One place that decides how much of the machine a training run may use, so no +stage has to guess. Import and call `claim()` at the top of any script that +trains something. + + from mlate import resources as res + res.claim() # sets thread counts, GPU fractions, env vars + print(res.BUDGET) + +Policy (set in config.py): + CPU 80% of logical cores + RAM 85% of physical memory + GPU 90% of each visible device's memory + +The CPU share is applied three ways, because they are enforced by different +layers and setting only one leaves the others unbounded: + - n_jobs on scikit-learn / XGBoost / LightGBM estimators + - OMP/MKL/OPENBLAS thread counts, which control the BLAS underneath them + - torch's intra-op and inter-op thread pools + +Oversubscription warning: if you run N estimators in parallel and each is given +N_JOBS threads, you request N x N_JOBS threads and the machine thrashes. Use +`inner_jobs(n_parallel)` to divide the budget when nesting parallelism. +""" + +from __future__ import annotations + +import os +from dataclasses import dataclass, field + +from mlate import config as cfg + + +@dataclass(frozen=True) +class Budget: + cpu_total: int + n_jobs: int + ram_total_gib: float + ram_limit_gib: float + gpus: list[str] = field(default_factory=list) + gpu_fraction: float = cfg.GPU_MEMORY_FRACTION + + def __str__(self) -> str: + gpu = ", ".join(f"[{i}] {n}" for i, n in enumerate(self.gpus)) or "none" + return (f"CPU {self.n_jobs}/{self.cpu_total} threads | " + f"RAM {self.ram_limit_gib:.0f}/{self.ram_total_gib:.0f} GiB | " + f"GPU {self.gpu_fraction:.0%} of {gpu}") + + +def _detect() -> Budget: + try: + import psutil + ram = psutil.virtual_memory().total / 2 ** 30 + except Exception: + ram = 0.0 + + gpus: list[str] = [] + try: + import torch + if torch.cuda.is_available(): + gpus = [torch.cuda.get_device_properties(i).name + for i in range(torch.cuda.device_count())] + except Exception: + pass + + return Budget( + cpu_total=cfg.CPU_TOTAL, + n_jobs=cfg.N_JOBS, + ram_total_gib=ram, + ram_limit_gib=ram * cfg.RAM_FRACTION, + gpus=gpus, + ) + + +BUDGET = _detect() + + +def claim(verbose: bool = True) -> Budget: + """Apply the budget to this process. Call once, before importing heavy work.""" + threads = str(BUDGET.n_jobs) + for var in ("OMP_NUM_THREADS", "MKL_NUM_THREADS", "OPENBLAS_NUM_THREADS", + "NUMEXPR_NUM_THREADS", "VECLIB_MAXIMUM_THREADS"): + os.environ.setdefault(var, threads) + + try: + import torch + torch.set_num_threads(BUDGET.n_jobs) + torch.set_num_interop_threads(max(1, BUDGET.n_jobs // 8)) + if torch.cuda.is_available(): + for i in range(torch.cuda.device_count()): + torch.cuda.set_per_process_memory_fraction( + cfg.GPU_MEMORY_FRACTION, device=i) + torch.backends.cudnn.benchmark = True + torch.set_float32_matmul_precision("high") + except Exception: + pass + + if verbose: + print(f"compute budget: {BUDGET}") + return BUDGET + + +def single_thread(): + """ + Context manager forcing every native thread pool to one thread. + + Use this around any fit executed inside a worker process. Setting + OMP_NUM_THREADS and friends from inside the worker does NOT work, and the + failure is silent and expensive: OpenMP reads its environment when the + runtime initialises, which has already happened by the time a worker + function runs. Estimators that thread through OpenMP rather than through an + n_jobs argument - HistGradientBoosting above all, plus anything reaching + BLAS, such as MLP - therefore keep spawning one thread per core inside + every worker. + + Measured cost of getting this wrong: with 54 workers each spawning ~64 + OpenMP threads (3,456 threads on 64 cores), a HistGradientBoosting fit that + takes 1.46 s alone measured 246 s - a 160x slowdown, with mean, median and + p95 all equal, which is the signature of contention rather than of + expensive hyper-parameters. + + threadpoolctl reaches the pools at runtime through their C interfaces, so + it works regardless of import order. Estimators that expose n_jobs or + thread_count should still be passed 1 as well; this catches the ones that + do not. + """ + from threadpoolctl import threadpool_limits + return threadpool_limits(limits=1) + + +def inner_jobs(n_parallel: int) -> int: + """ + Threads per worker when running `n_parallel` jobs at once. + + Keeps total demand at the CPU budget instead of n_parallel times it. + """ + return max(1, BUDGET.n_jobs // max(1, n_parallel)) + + +def devices() -> list[str]: + """Torch device strings for every visible GPU, or CPU if there are none.""" + return [f"cuda:{i}" for i in range(len(BUDGET.gpus))] or ["cpu"] + + +def device_for(index: int) -> str: + """Round-robin a unit of work across the available GPUs.""" + d = devices() + return d[index % len(d)] diff --git a/mlate/scheduling.py b/mlate/scheduling.py new file mode 100644 index 0000000000000000000000000000000000000000..7865a416f51573812c1365cdf0be2d33bf32b349 --- /dev/null +++ b/mlate/scheduling.py @@ -0,0 +1,188 @@ +""" +Cost-aware scheduling for the tuning grid +========================================= + +The tuning grid is 33 models x 2 tasks x 2 protocols x 10 outer folds. Those +units are not remotely equal in cost: measured on a real fold, one fit ranges +from 0.004 s (Extra Tree) to 11.6 s (Stacking) - a factor of ~3,000. Handing +that to a worker pool in registry order wastes most of the machine, because the +expensive units are scattered and the pool drains into a long tail where one +worker finishes Stacking while fifty-one sit idle. + +Longest-processing-time-first +----------------------------- +Units are dispatched most-expensive-first. LPT is the classic greedy makespan +heuristic and is guaranteed within 4/3 of optimal for identical machines; more +usefully here, it bounds the tail, because the last thing dispatched is by +construction the cheapest thing left rather than possibly the most expensive. +The cheap units backfill behind the expensive ones instead of the reverse. + +Where the costs come from +------------------------- +Measured, not guessed - `04_machine_learning/calibrate.py` times one +single-threaded fit of every model on a real training fold and writes +fit_costs.xlsx. The registry's `cost` tag ("low"/"medium"/"high") is only a +coarse fallback for when that table is missing, and it is wrong often enough to +matter: the tag calls SVM (RBF) "high" at 0.81 s and CatBoost "medium" at +5.91 s. + +What the estimate does NOT capture +---------------------------------- +A tuning trial fits whatever hyper-parameters Optuna suggested, not the +registry's defaults, and the search spaces reach further than the defaults +(RandomForest to 500 trees, GradientBoosting to 400 x depth 20). The estimate is +therefore a lower bound with a roughly constant bias per model, which is all LPT +needs: ordering is preserved under a monotone transform, so the schedule is +unaffected even where the absolute predictions are low. Pruning cuts real cost +further and unevenly. Treat the totals as planning figures, not promises. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +import pandas as pd + +from mlate import config as cfg + +# Coarse fallback, seconds per fit, used only when fit_costs.xlsx is absent. +# Deliberately pessimistic: over-estimating a cheap model costs a little +# ordering accuracy, under-estimating an expensive one costs a long tail. +FALLBACK = {"low": 0.2, "medium": 2.0, "high": 6.0} + +# Inner-CV folds train on (k-1)/k of the outer training partition. +INNER_TRAIN_FRACTION = 0.9 + + +@dataclass(frozen=True) +class Unit: + """One (model, task, protocol, outer fold) job.""" + task: str + protocol: str + model: str + fold_index: int + fold_name: str = "" + est_seconds: float = 0.0 + + @property + def key(self) -> str: + return f"{self.task}__{self.protocol}__{self.model}__fold{self.fold_index}" + + @property + def safe_key(self) -> str: + """ + Filesystem-safe form of `key`. + + Model names carry spaces, parentheses, '+' and '->' ("Stacking + (RF+XGB+LR -> LR)"), none of which belong in a Windows filename. + """ + out = self.key + for ch in " ()+->/\\": + out = out.replace(ch, "_") + return "_".join(part for part in out.split("_") if part) + + +def load_fit_costs(path=None) -> dict[tuple[str, str], float]: + """ + Measured seconds per single-threaded fit, keyed by (task, model). + + Returns an empty mapping when the table has not been generated yet; callers + fall back to the registry's cost tag. + """ + path = path or (cfg.step_dir("04_machine_learning", "tables") + / "fit_costs.xlsx") + if not path.exists(): + return {} + df = pd.read_excel(path) + return {(str(r.task), str(r.model)): float(r.seconds) + for r in df.itertuples() if float(r.seconds) > 0} + + +def fit_seconds(model: str, task: str, costs: dict) -> float: + """Best available estimate of one fit, in seconds.""" + if (task, model) in costs: + return costs[(task, model)] + # Any task's measurement beats the cost tag. + same_model = [v for (t, m), v in costs.items() if m == model] + if same_model: + return sum(same_model) / len(same_model) + from mlate import models as zoo + tag = zoo.REGISTRY[model].cost if model in zoo.REGISTRY else "medium" + return FALLBACK.get(tag, FALLBACK["medium"]) + + +def estimate_unit(model: str, task: str, costs: dict, n_trials: int, + inner_folds: int) -> float: + """ + Seconds for one tuning unit: every trial fits once per inner fold, then the + winning configuration is refitted once on the whole outer training set. + """ + from mlate import search_spaces as ss + + per_fit = fit_seconds(model, task, costs) + if model in ss.BASELINES: + return per_fit # fitted once, never searched + if model in ss.META: + return per_fit # composed from tuned bases, one fit + return per_fit * (n_trials * inner_folds * INNER_TRAIN_FRACTION + 1.0) + + +def build_units(tasks, protocols, models, n_folds_by_cell: dict, + n_trials: int, inner_folds: int, + costs: dict | None = None) -> list[Unit]: + """Every unit in the grid, ordered longest-processing-time-first.""" + costs = load_fit_costs() if costs is None else costs + units = [ + Unit(task=task, protocol=protocol, model=model, fold_index=k, + est_seconds=estimate_unit(model, task, costs, n_trials, + inner_folds)) + for task in tasks + for protocol in protocols + for model in models + for k in range(n_folds_by_cell[(task, protocol)]) + ] + return sorted(units, key=lambda u: u.est_seconds, reverse=True) + + +def makespan(units: list[Unit], workers: int) -> dict: + """ + Simulate LPT onto `workers` identical machines. + + Reports the predicted wall-clock and how much of the pool the schedule + actually keeps busy, which is the number worth watching: a high total with + low utilisation means the grid is tail-bound and wants smaller units, not + more cores. + """ + if not units: + return {"n_units": 0, "core_seconds": 0.0, "makespan_seconds": 0.0, + "utilisation": 0.0, "workers": workers} + end = [0.0] * workers + for u in units: # already sorted longest-first + i = min(range(workers), key=lambda j: end[j]) + end[i] += u.est_seconds + total = sum(u.est_seconds for u in units) + span = max(end) + return { + "n_units": len(units), + "core_seconds": total, + "core_hours": total / 3600.0, + "makespan_seconds": span, + "makespan_hours": span / 3600.0, + "utilisation": total / (span * workers) if span else 0.0, + "workers": workers, + } + + +def plan_table(units: list[Unit]) -> pd.DataFrame: + """Per-model cost roll-up, for the run log and the supplement.""" + df = pd.DataFrame([{"model": u.model, "task": u.task, + "protocol": u.protocol, "est_seconds": u.est_seconds} + for u in units]) + out = (df.groupby("model") + .agg(units=("est_seconds", "size"), + est_seconds_each=("est_seconds", "mean"), + est_core_hours=("est_seconds", lambda s: s.sum() / 3600.0)) + .sort_values("est_core_hours", ascending=False) + .reset_index()) + out["share_of_total"] = out["est_core_hours"] / out["est_core_hours"].sum() + return out diff --git a/mlate/search_spaces.py b/mlate/search_spaces.py new file mode 100644 index 0000000000000000000000000000000000000000..964755a73d712969df8408b40a4f1e0526f07b1b --- /dev/null +++ b/mlate/search_spaces.py @@ -0,0 +1,735 @@ +""" +Hyper-parameter search spaces +============================= + +One Optuna space per entry in the registry, so the tuned benchmark covers the +same 33 models as the untuned one and a model cannot be searched over different +ranges in two scripts. + +Provenance +---------- +Twenty-seven spaces are transcribed from the submitted pipeline +(`classification_pipeline.py`), unchanged in range, so the tuned numbers here +are comparable with the ones already reported rather than with a space of our +own choosing. Four are new because the registry carries models the submitted +pipeline did not: SVC with an RBF and a polynomial kernel, and the two +meta-ensembles. Two are deliberately absent - see BASELINES below. + +Constrained variants +-------------------- +The registry carries paired entries that differ by exactly one setting: +Logistic Regression / (balanced), Random Forest / (balanced), and k-NN / +(distance weighted). The submitted pipeline searched `class_weight` and +`weights` freely, which would collapse each pair into two draws from one space +and report the same search twice under different names. Here the distinguishing +setting is PINNED per variant and removed from the space, so each pair is two +constrained searches whose difference is the thing the pair exists to show. + +BASELINES +--------- +The two dummies are not tunable. `strategy` is not a hyper-parameter of +DummyClassifier, it is the definition of which baseline it is - searching it +would let "Dummy (majority)" silently become "Dummy (stratified)" and destroy +the constant-prediction reference every accuracy figure is read against. They +are carried into the tuned tables at their untuned scores. + +Conditional parameters +---------------------- +Several estimators reject otherwise-valid combinations (RandomForest's +`max_samples` without bootstrap, CatBoost's `bagging_temperature` outside +Bayesian bootstrap). Optuna suggests the parameter regardless, so the trial is +reproducible from its params dict, and `build_tuned` drops it at construction +time. Resolving it inside the space instead would make the recorded params +disagree with the object that was actually fitted. +""" + +from __future__ import annotations + +from typing import Callable + +from sklearn.discriminant_analysis import (LinearDiscriminantAnalysis, + QuadraticDiscriminantAnalysis) +from sklearn.decomposition import PCA +from sklearn.dummy import DummyClassifier +from sklearn.ensemble import (AdaBoostClassifier, BaggingClassifier, + ExtraTreesClassifier, GradientBoostingClassifier, + HistGradientBoostingClassifier, + RandomForestClassifier, StackingClassifier, + VotingClassifier) +from sklearn.linear_model import (LogisticRegression, + PassiveAggressiveClassifier, Perceptron, + RidgeClassifier, SGDClassifier) +from sklearn.naive_bayes import BernoulliNB, GaussianNB +from sklearn.neighbors import KNeighborsClassifier, NearestCentroid +from sklearn.neural_network import MLPClassifier +from sklearn.pipeline import Pipeline +from sklearn.svm import SVC, LinearSVC +from sklearn.tree import DecisionTreeClassifier, ExtraTreeClassifier + +from mlate import config as cfg + +SEED = cfg.RANDOM_STATE + +# Models carried through untuned. See BASELINES in the module docstring. +BASELINES = ("Dummy (majority)", "Dummy (stratified)") + +# Built from other models' tuned parameters rather than from a space of their +# own; the tuner runs them in a second wave. See build_meta(). +META = ("Soft Voting (RF+XGB+LR)", "Stacking (RF+XGB+LR -> LR)") + +# Tuples do not survive Optuna's storage round-trip, so layer shapes are +# searched as strings and mapped back at construction. Same device the +# submitted pipeline used. +_MLP_LAYERS = { + "64": (64,), "128": (128,), "256": (256,), + "128_64": (128, 64), "256_128": (256, 128), "256_128_64": (256, 128, 64), +} + + +# ── spaces ────────────────────────────────────────────────────────────────── +# Each returns the raw params dict for one trial. Nothing is constructed here. + +def _logistic(t) -> dict: + return { + "penalty": t.suggest_categorical("penalty", ["l2", None]), + "tol": t.suggest_float("tol", 1e-5, 1e-2, log=True), + "C": t.suggest_float("C", 0.01, 100.0, log=True), + "fit_intercept": t.suggest_categorical("fit_intercept", [True, False]), + "solver": t.suggest_categorical("solver", ["lbfgs", "newton-cg", "saga"]), + "max_iter": t.suggest_int("max_iter", 200, 2000), + } + + +def _ridge(t) -> dict: + return { + "alpha": t.suggest_float("alpha", 0.01, 100.0, log=True), + "fit_intercept": t.suggest_categorical("fit_intercept", [True, False]), + "max_iter": t.suggest_int("max_iter", 100, 2000), + "tol": t.suggest_float("tol", 1e-5, 1e-2, log=True), + "solver": t.suggest_categorical("solver", ["auto", "sparse_cg", "sag", + "saga"]), + } + + +def _sgd(t) -> dict: + # loss is pinned to hinge: the registry entry is "SGD (hinge)", and letting + # the search pick log_loss would turn it into a different classifier. + return { + "penalty": t.suggest_categorical("penalty", ["l2", "l1", "elasticnet"]), + "alpha": t.suggest_float("alpha", 1e-6, 1e-1, log=True), + "l1_ratio": t.suggest_float("l1_ratio", 0.0, 1.0), + "fit_intercept": t.suggest_categorical("fit_intercept", [True, False]), + "max_iter": t.suggest_int("max_iter", 200, 3000), + "tol": t.suggest_float("tol", 1e-5, 1e-2, log=True), + "learning_rate": t.suggest_categorical( + "learning_rate", ["constant", "optimal", "invscaling", "adaptive"]), + "eta0": t.suggest_float("eta0", 1e-5, 1.0, log=True), + "early_stopping": t.suggest_categorical("early_stopping", [True, False]), + "validation_fraction": t.suggest_float("validation_fraction", 0.05, 0.3), + "n_iter_no_change": t.suggest_int("n_iter_no_change", 1, 10), + } + + +def _passive_aggressive(t) -> dict: + return { + "C": t.suggest_float("C", 1e-4, 1e4, log=True), + "fit_intercept": t.suggest_categorical("fit_intercept", [True, False]), + "max_iter": t.suggest_int("max_iter", 100, 3000), + "tol": t.suggest_float("tol", 1e-5, 1e-1, log=True), + "early_stopping": t.suggest_categorical("early_stopping", [True, False]), + "validation_fraction": t.suggest_float("validation_fraction", 0.05, 0.3), + "n_iter_no_change": t.suggest_int("n_iter_no_change", 1, 10), + "shuffle": t.suggest_categorical("shuffle", [True, False]), + "loss": t.suggest_categorical("loss", ["hinge", "squared_hinge"]), + "class_weight": t.suggest_categorical("class_weight", [None, "balanced"]), + "average": t.suggest_categorical("average", [False, True]), + } + + +def _perceptron(t) -> dict: + return { + "penalty": t.suggest_categorical("penalty", + [None, "l2", "l1", "elasticnet"]), + "alpha": t.suggest_float("alpha", 1e-7, 1e-1, log=True), + "l1_ratio": t.suggest_float("l1_ratio", 0.0, 1.0), + "fit_intercept": t.suggest_categorical("fit_intercept", [True, False]), + "max_iter": t.suggest_int("max_iter", 100, 3000), + "tol": t.suggest_float("tol", 1e-5, 1e-1, log=True), + "shuffle": t.suggest_categorical("shuffle", [True, False]), + "eta0": t.suggest_float("eta0", 1e-3, 10, log=True), + "early_stopping": t.suggest_categorical("early_stopping", [True, False]), + "validation_fraction": t.suggest_float("validation_fraction", 0.05, 0.3), + "n_iter_no_change": t.suggest_int("n_iter_no_change", 1, 10), + "class_weight": t.suggest_categorical("class_weight", [None, "balanced"]), + } + + +def _lda(t) -> dict: + # n_components is capped at n_classes-1 by sklearn; it is searched over the + # submitted range and clipped at construction against the fold's classes. + return { + "shrinkage": t.suggest_categorical("shrinkage", + [None, "auto", 0.1, 0.5, 0.9]), + "n_components": t.suggest_int("n_components", 1, 10), + "tol": t.suggest_float("tol", 1e-5, 1e-2, log=True), + } + + +def _qda(t) -> dict: + # The registry fits QDA behind a PCA because the smallest class has fewer + # samples than the 153 features, so the per-class covariances are singular. + # The PCA width is therefore part of the model and is searched with it. + return { + "pca__n_components": t.suggest_int("pca__n_components", 5, 60), + "reg_param": t.suggest_float("reg_param", 0.0, 1.0), + "store_covariance": t.suggest_categorical("store_covariance", + [True, False]), + "tol": t.suggest_float("tol", 1e-7, 1e-1, log=True), + } + + +def _gaussian_nb(t) -> dict: + return {"var_smoothing": t.suggest_float("var_smoothing", 1e-11, 1e-5, + log=True)} + + +def _bernoulli_nb(t) -> dict: + return { + "alpha": t.suggest_float("alpha", 1e-10, 10.0, log=True), + "binarize": t.suggest_float("binarize", 0.0, 1.0), + "fit_prior": t.suggest_categorical("fit_prior", [True, False]), + } + + +def _knn(t) -> dict: + return { + "n_neighbors": t.suggest_int("n_neighbors", 1, 50), + "algorithm": t.suggest_categorical("algorithm", + ["auto", "ball_tree", "kd_tree", + "brute"]), + "leaf_size": t.suggest_int("leaf_size", 10, 100), + "p": t.suggest_int("p", 1, 5), + "metric": t.suggest_categorical("metric", ["minkowski", "euclidean", + "manhattan"]), + } + + +def _nearest_centroid(t) -> dict: + return { + "metric": t.suggest_categorical("metric", ["euclidean", "manhattan"]), + "shrink_threshold": t.suggest_float("shrink_threshold", 0.0, 1.0), + } + + +def _svc_rbf(t) -> dict: + # New. The submitted pipeline carried NuSVC and LinearSVC but no C-SVC with + # an RBF kernel, which is what the registry entry is. Ranges follow the + # NuSVC entry where they correspond. + return { + "C": t.suggest_float("C", 0.01, 100.0, log=True), + "gamma": t.suggest_categorical("gamma", ["scale", "auto"]), + "tol": t.suggest_float("tol", 1e-4, 1e-2, log=True), + "shrinking": t.suggest_categorical("shrinking", [True, False]), + "class_weight": t.suggest_categorical("class_weight", [None, "balanced"]), + "decision_function_shape": t.suggest_categorical( + "decision_function_shape", ["ovo", "ovr"]), + } + + +def _svc_poly(t) -> dict: + space = _svc_rbf(t) + space["degree"] = t.suggest_int("degree", 2, 5) + space["coef0"] = t.suggest_float("coef0", 0.0, 1.0) + return space + + +def _linear_svc(t) -> dict: + return { + "loss": t.suggest_categorical("loss", ["hinge", "squared_hinge"]), + "tol": t.suggest_float("tol", 1e-5, 1e-2, log=True), + "C": t.suggest_float("C", 0.01, 100.0, log=True), + "fit_intercept": t.suggest_categorical("fit_intercept", [True, False]), + "intercept_scaling": t.suggest_float("intercept_scaling", 0.1, 10.0, + log=True), + "max_iter": t.suggest_int("max_iter", 500, 5000), + "class_weight": t.suggest_categorical("class_weight", [None, "balanced"]), + } + + +def _decision_tree(t) -> dict: + return { + "criterion": t.suggest_categorical("criterion", + ["gini", "entropy", "log_loss"]), + "splitter": t.suggest_categorical("splitter", ["best", "random"]), + "max_depth": t.suggest_int("max_depth", 1, 50), + "min_samples_split": t.suggest_int("min_samples_split", 2, 20), + "min_samples_leaf": t.suggest_int("min_samples_leaf", 1, 20), + "max_features": t.suggest_categorical("max_features", + [None, "sqrt", "log2"]), + "ccp_alpha": t.suggest_float("ccp_alpha", 0.0, 0.05), + } + + +def _extra_tree(t) -> dict: + # Deliberately not _decision_tree() minus splitter: calling that and + # discarding the value still SUGGESTS splitter, so Optuna records a + # parameter that cannot affect the fit and the sampler wastes a dimension + # modelling it. ExtraTreeClassifier splits at random by definition. + return { + "criterion": t.suggest_categorical("criterion", + ["gini", "entropy", "log_loss"]), + "max_depth": t.suggest_int("max_depth", 1, 50), + "min_samples_split": t.suggest_int("min_samples_split", 2, 20), + "min_samples_leaf": t.suggest_int("min_samples_leaf", 1, 20), + "max_features": t.suggest_categorical("max_features", + [None, "sqrt", "log2"]), + "ccp_alpha": t.suggest_float("ccp_alpha", 0.0, 0.05), + } + + +def _random_forest(t) -> dict: + # class_weight is absent on purpose: it is what separates the two RF + # entries in the registry and is pinned per variant in build_tuned. + return { + "n_estimators": t.suggest_int("n_estimators", 100, 500, step=50), + "criterion": t.suggest_categorical("criterion", + ["gini", "entropy", "log_loss"]), + "max_depth": t.suggest_int("max_depth", 3, 40), + "min_samples_split": t.suggest_int("min_samples_split", 2, 10), + "min_samples_leaf": t.suggest_int("min_samples_leaf", 1, 10), + "min_weight_fraction_leaf": t.suggest_float("min_weight_fraction_leaf", + 0.0, 0.2), + "max_features": t.suggest_categorical("max_features", + ["sqrt", "log2", None]), + "bootstrap": t.suggest_categorical("bootstrap", [True, False]), + "ccp_alpha": t.suggest_float("ccp_alpha", 0.0, 0.1), + "max_samples": t.suggest_categorical("max_samples", + [None, 0.7, 0.8, 0.9]), + } + + +def _extra_trees(t) -> dict: + return { + "n_estimators": t.suggest_int("n_estimators", 50, 500), + "criterion": t.suggest_categorical("criterion", + ["gini", "entropy", "log_loss"]), + "max_depth": t.suggest_int("max_depth", 3, 60), + "min_samples_split": t.suggest_int("min_samples_split", 2, 20), + "min_samples_leaf": t.suggest_int("min_samples_leaf", 1, 20), + "max_features": t.suggest_categorical("max_features", + ["sqrt", "log2", None]), + "bootstrap": t.suggest_categorical("bootstrap", [True, False]), + "ccp_alpha": t.suggest_float("ccp_alpha", 0.0, 0.05), + } + + +def _bagging(t) -> dict: + # The registry bags decision trees, so the base tree's depth is part of the + # model and is searched alongside the ensemble settings. + return { + "n_estimators": t.suggest_int("n_estimators", 10, 200), + "max_samples": t.suggest_float("max_samples", 0.5, 1.0), + "max_features": t.suggest_float("max_features", 0.5, 1.0), + "bootstrap": t.suggest_categorical("bootstrap", [True, False]), + "estimator__max_depth": t.suggest_int("estimator__max_depth", 3, 30), + "estimator__min_samples_leaf": t.suggest_int( + "estimator__min_samples_leaf", 1, 20), + } + + +def _adaboost(t) -> dict: + return { + "n_estimators": t.suggest_int("n_estimators", 50, 300), + "learning_rate": t.suggest_float("learning_rate", 0.01, 2.0, log=True), + } + + +def _gradient_boosting(t) -> dict: + return { + "learning_rate": t.suggest_float("learning_rate", 0.01, 1.0, log=True), + "n_estimators": t.suggest_int("n_estimators", 50, 400), + "subsample": t.suggest_float("subsample", 0.5, 1.0), + "criterion": t.suggest_categorical("criterion", + ["friedman_mse", "squared_error"]), + "min_samples_split": t.suggest_int("min_samples_split", 2, 20), + "min_samples_leaf": t.suggest_int("min_samples_leaf", 1, 20), + "max_depth": t.suggest_int("max_depth", 3, 20), + "max_features": t.suggest_categorical("max_features", + ["sqrt", "log2", None]), + "ccp_alpha": t.suggest_float("ccp_alpha", 0.0, 0.05), + } + + +def _hist_gradient_boosting(t) -> dict: + return { + "learning_rate": t.suggest_float("learning_rate", 0.01, 1.0, log=True), + "max_iter": t.suggest_int("max_iter", 50, 500), + "max_leaf_nodes": t.suggest_int("max_leaf_nodes", 10, 100), + "max_depth": t.suggest_int("max_depth", 3, 30), + "min_samples_leaf": t.suggest_int("min_samples_leaf", 1, 50), + "l2_regularization": t.suggest_float("l2_regularization", 0.0, 1.0), + "max_bins": t.suggest_int("max_bins", 50, 255), + "early_stopping": t.suggest_categorical("early_stopping", [True, False]), + "n_iter_no_change": t.suggest_int("n_iter_no_change", 5, 20), + } + + +def _xgboost(t) -> dict: + return { + "n_estimators": t.suggest_int("n_estimators", 50, 500), + "max_depth": t.suggest_int("max_depth", 3, 12), + "learning_rate": t.suggest_float("learning_rate", 0.005, 0.3, log=True), + "gamma": t.suggest_float("gamma", 0.0, 5.0), + "min_child_weight": t.suggest_float("min_child_weight", 1.0, 10.0), + "subsample": t.suggest_float("subsample", 0.5, 1.0), + "colsample_bytree": t.suggest_float("colsample_bytree", 0.5, 1.0), + "reg_alpha": t.suggest_float("reg_alpha", 0.0, 10.0), + "reg_lambda": t.suggest_float("reg_lambda", 0.0, 10.0), + } + + +def _lightgbm(t) -> dict: + return { + "boosting_type": t.suggest_categorical("boosting_type", + ["gbdt", "dart", "rf"]), + "num_leaves": t.suggest_int("num_leaves", 31, 256), + "max_depth": t.suggest_int("max_depth", 3, 50), + "learning_rate": t.suggest_float("learning_rate", 0.005, 0.3, log=True), + "n_estimators": t.suggest_int("n_estimators", 50, 500), + "min_child_weight": t.suggest_float("min_child_weight", 1e-3, 10.0), + "min_child_samples": t.suggest_int("min_child_samples", 5, 50), + "subsample": t.suggest_float("subsample", 0.5, 1.0), + "subsample_freq": t.suggest_int("subsample_freq", 0, 10), + "colsample_bytree": t.suggest_float("colsample_bytree", 0.5, 1.0), + "reg_alpha": t.suggest_float("reg_alpha", 0.0, 10.0), + "reg_lambda": t.suggest_float("reg_lambda", 0.0, 10.0), + } + + +def _catboost(t) -> dict: + # iterations capped at 500 rather than the submitted pipeline's 1,000. + # Measured: CatBoost is 48% of the entire tuning grid on its own, and its + # cost is close to linear in iterations, so the upper half of that range + # was buying most of the compute bill. An 800-1,000 iteration model at + # depth 10 on 2,378 rows is also comfortably into overfitting, so the + # configurations being dropped are ones a fold would not have selected. + return { + "iterations": t.suggest_int("iterations", 100, 500), + "learning_rate": t.suggest_float("learning_rate", 0.005, 0.3, log=True), + "depth": t.suggest_int("depth", 3, 10), + "l2_leaf_reg": t.suggest_float("l2_leaf_reg", 1.0, 10.0), + "bagging_temperature": t.suggest_float("bagging_temperature", 0.0, 1.0), + "random_strength": t.suggest_float("random_strength", 0.0, 10.0), + "colsample_bylevel": t.suggest_float("colsample_bylevel", 0.5, 1.0), + # boosting_type is fixed to Plain rather than searched over + # {Plain, Ordered}. Measured on a real fold at 300 iterations, depth 6: + # Plain 2.65 s, Ordered 16.10 s - a 6.1x penalty on a categorical drawn + # half the time, which by itself made CatBoost roughly half of the + # entire tuning grid. Ordered boosting is also unsupported on GPU under + # a multiclass loss, so it forecloses that option too. The trade is a + # narrower search than the submitted pipeline's, and it must be stated + # in the Methods rather than left implicit. + "bootstrap_type": t.suggest_categorical("bootstrap_type", + ["Bayesian", "Bernoulli"]), + } + + +def _mlp(t) -> dict: + return { + "hidden_layer_sizes": t.suggest_categorical( + "hidden_layer_sizes", list(_MLP_LAYERS)), + "activation": t.suggest_categorical("activation", + ["relu", "tanh", "logistic"]), + # lbfgs is excluded from the submitted pipeline's {adam, sgd, lbfgs}. + # It is a full-batch quasi-Newton solver that scikit-learn recommends + # only for small datasets, and on this 2,378 x 153 problem it does not + # converge: every sampled draw exhausted max_iter, emitting + # "lbfgs failed to converge after 965 iteration(s)". Those draws burn + # the entire iteration budget to return a non-converged model, which + # made MLP the single most expensive entry in the grid. Removing a + # solver that never converges removes configurations that could not + # have been selected on merit. + "solver": t.suggest_categorical("solver", ["adam", "sgd"]), + "alpha": t.suggest_float("alpha", 1e-6, 1e-1, log=True), + "learning_rate": t.suggest_categorical( + "learning_rate", ["constant", "invscaling", "adaptive"]), + "learning_rate_init": t.suggest_float("learning_rate_init", 1e-5, 1e-1, + log=True), + "max_iter": t.suggest_int("max_iter", 200, 1000), + "tol": t.suggest_float("tol", 1e-6, 1e-3, log=True), + "early_stopping": t.suggest_categorical("early_stopping", [True, False]), + } + + +SPACES: dict[str, Callable] = { + "Logistic Regression": _logistic, + "Logistic Regression (balanced)": _logistic, + "Ridge Classifier": _ridge, + "SGD (hinge)": _sgd, + "Passive Aggressive": _passive_aggressive, + "Perceptron": _perceptron, + "Linear Discriminant": _lda, + "Quadratic Discriminant": _qda, + "Gaussian Naive Bayes": _gaussian_nb, + "Bernoulli Naive Bayes": _bernoulli_nb, + "k-Nearest Neighbours": _knn, + "k-NN (distance weighted)": _knn, + "Nearest Centroid": _nearest_centroid, + "SVM (RBF)": _svc_rbf, + "SVM (polynomial)": _svc_poly, + "Linear SVM": _linear_svc, + "Decision Tree": _decision_tree, + "Extra Tree": _extra_tree, + "Random Forest": _random_forest, + "Random Forest (balanced)": _random_forest, + "Extra Trees": _extra_trees, + "Bagged Trees": _bagging, + "AdaBoost": _adaboost, + "Gradient Boosting": _gradient_boosting, + "Hist Gradient Boosting": _hist_gradient_boosting, + "XGBoost": _xgboost, + "LightGBM": _lightgbm, + "CatBoost": _catboost, + "MLP (256-128)": _mlp, +} + +# What each paired variant pins, and therefore does not search. +PINNED: dict[str, dict] = { + "Logistic Regression": {"class_weight": None}, + "Logistic Regression (balanced)": {"class_weight": "balanced"}, + "Random Forest": {"class_weight": None}, + "Random Forest (balanced)": {"class_weight": "balanced_subsample"}, + "k-Nearest Neighbours": {"weights": "uniform"}, + "k-NN (distance weighted)": {"weights": "distance"}, +} + + +def suggest(name: str, trial) -> dict: + """Draw one trial's parameters for `name`.""" + if name not in SPACES: + raise KeyError(f"no search space for {name!r}") + return SPACES[name](trial) + + +# ── construction ──────────────────────────────────────────────────────────── + +def build_tuned(name: str, params: dict, n_jobs: int = 1, + n_classes: int | None = None): + """ + Build one estimator from a trial's parameters. + + Conditional parameters that the estimator would reject are dropped here + rather than in the space, so the recorded params always describe the trial + that was actually run. + """ + p = dict(params) + p.update(PINNED.get(name, {})) + + if name == "Logistic Regression" or name == "Logistic Regression (balanced)": + # saga is the only solver in the space that accepts penalty=None with + # every configuration; lbfgs and newton-cg accept it too, so nothing is + # dropped, but l1_ratio never appears and elasticnet is not offered. + return LogisticRegression(random_state=SEED, **p) + + if name == "Ridge Classifier": + return RidgeClassifier(random_state=SEED, **p) + + if name == "SGD (hinge)": + if not p.get("early_stopping", False): + p.pop("validation_fraction", None) + if p.get("penalty") != "elasticnet": + p.pop("l1_ratio", None) + return SGDClassifier(loss="hinge", random_state=SEED, **p) + + if name == "Passive Aggressive": + if not p.get("early_stopping", False): + p.pop("validation_fraction", None) + return PassiveAggressiveClassifier(random_state=SEED, **p) + + if name == "Perceptron": + if not p.get("early_stopping", False): + p.pop("validation_fraction", None) + if p.get("penalty") != "elasticnet": + p.pop("l1_ratio", None) + return Perceptron(random_state=SEED, **p) + + if name == "Linear Discriminant": + # n_components cannot exceed n_classes - 1. + if n_classes is not None: + p["n_components"] = min(p["n_components"], max(1, n_classes - 1)) + # shrinkage is only supported by the lsqr and eigen solvers. + solver = "lsqr" if p.get("shrinkage") is not None else "svd" + return LinearDiscriminantAnalysis(solver=solver, **p) + + if name == "Quadratic Discriminant": + n_comp = p.pop("pca__n_components") + return Pipeline([ + ("pca", PCA(n_components=n_comp, random_state=SEED)), + ("qda", QuadraticDiscriminantAnalysis(**p)), + ]) + + if name == "Gaussian Naive Bayes": + return GaussianNB(**p) + + if name == "Bernoulli Naive Bayes": + return BernoulliNB(**p) + + if name in ("k-Nearest Neighbours", "k-NN (distance weighted)"): + # p is only meaningful for minkowski; sklearn ignores it otherwise but + # warns, and kd_tree/ball_tree reject some metric choices. + if p.get("metric") != "minkowski": + p.pop("p", None) + return KNeighborsClassifier(n_jobs=n_jobs, **p) + + if name == "Nearest Centroid": + return NearestCentroid(**p) + + # max_iter is a guard, not a tuned parameter. scikit-learn's SVC defaults + # to max_iter=-1, i.e. unbounded libsvm iterations, and a polynomial kernel + # at high C and degree on 2,116 x 153 non-separable rows can effectively + # never converge - one such draw ran for SEVENTEEN HOURS on a single core + # before this cap existed. The submitted pipeline capped its NuSVC at + # max_iter <= 2000 for the same reason; dropping that guard was an error. + # + # 200,000 is chosen to be far above what a converging fit needs (typical + # draws here finish in a few seconds) while bounding the pathological ones + # to about a minute. A capped fit emits a ConvergenceWarning and returns + # its current solution, which for a trial that was never going to be + # selected is the correct outcome. cache_size is raised from the 200 MB + # default because the kernel cache is what libsvm spends its time missing. + if name == "SVM (RBF)": + return SVC(kernel="rbf", probability=True, max_iter=200_000, + cache_size=1000, random_state=SEED, **p) + + if name == "SVM (polynomial)": + return SVC(kernel="poly", probability=True, max_iter=200_000, + cache_size=1000, random_state=SEED, **p) + + if name == "Linear SVM": + # hinge is incompatible with the primal formulation, and dual='auto' + # resolves it; kept explicit so the combination cannot raise. + return LinearSVC(dual="auto", random_state=SEED, **p) + + if name == "Decision Tree": + return DecisionTreeClassifier(random_state=SEED, **p) + + if name == "Extra Tree": + return ExtraTreeClassifier(random_state=SEED, **p) + + if name in ("Random Forest", "Random Forest (balanced)"): + if not p.get("bootstrap", True): + p["max_samples"] = None # only valid when bootstrapping + return RandomForestClassifier(n_jobs=n_jobs, random_state=SEED, + oob_score=False, **p) + + if name == "Extra Trees": + return ExtraTreesClassifier(n_jobs=n_jobs, random_state=SEED, **p) + + if name == "Bagged Trees": + depth = p.pop("estimator__max_depth") + leaf = p.pop("estimator__min_samples_leaf") + return BaggingClassifier( + estimator=DecisionTreeClassifier(max_depth=depth, + min_samples_leaf=leaf, + random_state=SEED), + n_jobs=n_jobs, random_state=SEED, **p) + + if name == "AdaBoost": + # No `algorithm` argument: SAMME.R was deprecated and the parameter + # removed in scikit-learn 1.6, so the submitted pipeline's + # algorithm="SAMME" is now a TypeError rather than a no-op. SAMME is + # the only remaining behaviour, so dropping it changes nothing. + return AdaBoostClassifier(random_state=SEED, **p) + + if name == "Gradient Boosting": + return GradientBoostingClassifier(random_state=SEED, **p) + + if name == "Hist Gradient Boosting": + if not p.get("early_stopping", False): + p.pop("n_iter_no_change", None) + return HistGradientBoostingClassifier(random_state=SEED, **p) + + if name == "XGBoost": + from xgboost import XGBClassifier + return XGBClassifier(tree_method="hist", n_jobs=n_jobs, + random_state=SEED, verbosity=0, + eval_metric="mlogloss", **p) + + if name == "LightGBM": + # dart ignores subsample_freq, and rf requires bagging to be enabled. + if p.get("boosting_type") == "rf": + p["subsample_freq"] = max(1, p.get("subsample_freq", 1)) + p["subsample"] = min(p.get("subsample", 0.9), 0.99) + return __import__("lightgbm").LGBMClassifier( + n_jobs=n_jobs, random_state=SEED, verbose=-1, **p) + + if name == "CatBoost": + from catboost import CatBoostClassifier + if p.get("bootstrap_type") != "Bayesian": + p.pop("bagging_temperature", None) + return CatBoostClassifier(boosting_type="Plain", thread_count=n_jobs, + random_seed=SEED, verbose=0, + allow_writing_files=False, **p) + + if name == "MLP (256-128)": + p["hidden_layer_sizes"] = _MLP_LAYERS[str(p["hidden_layer_sizes"])] + if not p.get("early_stopping", False): + p.pop("n_iter_no_change", None) + return MLPClassifier(random_state=SEED, **p) + + raise KeyError(f"no builder for {name!r}") + + +def build_meta(name: str, tuned: dict, n_jobs: int = 1): + """ + Build a meta-ensemble from the tuned parameters of its base learners. + + `tuned` maps base-model registry names to their best parameters for THIS + fold. Composing from already-tuned bases rather than searching the + ensemble's own space keeps the comparison honest: the ensemble is the same + three learners the individual rows report, combined, so any gain it shows + is attributable to combination rather than to a larger search budget. It + also means the ensemble inherits the fold's nested-CV discipline, since the + base parameters were selected inside this fold's training partition. + + A base whose tuning failed falls back to the registry's untuned defaults, so + a single failure degrades the ensemble rather than removing it. + """ + from mlate import models as zoo + + def base(model_name: str, key: str): + params = tuned.get(model_name) + if params: + try: + return (key, build_tuned(model_name, params, n_jobs=n_jobs)) + except Exception: + pass + return (key, zoo.build(model_name, n_jobs=n_jobs)) + + estimators = [base("Random Forest", "rf"), + base("XGBoost", "xgb"), + base("Logistic Regression", "lr")] + + if name == "Soft Voting (RF+XGB+LR)": + return VotingClassifier(estimators=estimators, voting="soft", n_jobs=1) + if name == "Stacking (RF+XGB+LR -> LR)": + return StackingClassifier( + estimators=estimators, + final_estimator=LogisticRegression(max_iter=3000, + random_state=SEED), + cv=5, n_jobs=1, passthrough=False) + raise KeyError(f"not a meta-ensemble: {name!r}") + + +def coverage() -> dict: + """Which registry entries are searched, composed, or carried untuned.""" + from mlate import models as zoo + names = zoo.names() + return { + "searched": [n for n in names if n in SPACES], + "composed": [n for n in names if n in META], + "untuned": [n for n in names if n in BASELINES], + "unhandled": [n for n in names + if n not in SPACES and n not in META + and n not in BASELINES], + } diff --git a/mlate/serving.py b/mlate/serving.py new file mode 100644 index 0000000000000000000000000000000000000000..69b21faee9fe6869ed5881695e3e2a5d762848d6 --- /dev/null +++ b/mlate/serving.py @@ -0,0 +1,329 @@ +""" +Uniform loading of every deployable model, for inference +======================================================== + +The three families are trained and stored differently and predict differently. +A conventional classifier is a pickled fitted estimator. A deep network is a +state dictionary that must be poured back into a rebuilt architecture. A +foundation model is an in-context learner with no fitted parameters at all: the +checkpoint is fixed and pre-trained, and what the study "fitted" is the corpus +it conditions on, so the artefact stores that context rather than weights. + +Serving them through one interface is what lets the application offer all three +in a single menu, and lets the optimiser score candidates without knowing which +kind it is holding. + +Batching is the reason this module exposes `predict_proba` on a matrix rather +than a row. The cost of a foundation model is dominated by the pass over its +2,646 context rows, which is paid once per call however many candidates are +scored in it, so a batch of 32 costs what a batch of 1 costs. Measured on this +corpus: + + one row 32 rows per candidate in a batch + Bagged Trees 83 ms 80 ms 2.5 ms + TabICL (GPU) 997 ms 1 040 ms 33 ms + TabICL (CPU) 17 000 ms 16 900 ms 530 ms + +Scoring candidates singly, as the previous release did, therefore made the +foundation models appear unusable when they are not: it charged the context +pass once per candidate instead of once per batch. +""" + +from __future__ import annotations + +import json +import os +from dataclasses import dataclass, field +from pathlib import Path +from typing import Callable + +import joblib +import numpy as np + +from mlate import config as cfg + +# The exact TabPFN checkpoint the study used, so a served prediction matches a +# reported one. See 05_deep_learning/foundation_models.py for why this is a +# filename rather than a version string. +TABPFN_CHECKPOINT = "tabpfn-v2.6-classifier-v2.6_default.ckpt" +N_ESTIMATORS = 8 + +FAMILY_DIRS = {"ml": ("classifiers", True), "dl": ("deep", True), + "foundation": ("foundation", False)} + +# What a user is choosing between, in time rather than in architecture. Taken +# from the measurements in the module docstring and stated per 150-trial search +# over both targets, batched, on a CPU host. The application shows the real +# elapsed time once a search has run; this is only what it can say beforehand. +COST_HINT = {"ml": "fast, a few seconds", + "dl": "fast, a few seconds", + "foundation": "slow on CPU, roughly 3 minutes"} + + +def device() -> str: + """ + CUDA when it is genuinely available, CPU otherwise. + + Resolved once here rather than left to each library's own default, because + the two disagree: TabPFN refuses a CPU run with more than 1,000 context + rows unless told to proceed, while TabICL simply runs and is twenty times + slower than on a GPU. A single answer keeps the menu's cost estimate honest + for whichever host the application is deployed to. + """ + try: + import torch + return "cuda" if torch.cuda.is_available() else "cpu" + except Exception: + return "cpu" + + +# Keys a TabPFN token may arrive under. Read from the environment only - a +# Hugging Face Space supplies it as a secret, a local checkout as an +# uncommitted .env - and never written to a source file or logged. A missing +# token is not an error: the package then uses locally downloaded weights. +TABPFN_TOKEN_KEYS = ("TABPFN_TOKEN", "TABPFN_ACCESS_TOKEN", "PRIORLABS_TOKEN", + "PRIOR_LABS_TOKEN") + + +def _apply_tabpfn_token() -> bool: + for key in TABPFN_TOKEN_KEYS: + value = os.environ.get(key) + if value: + os.environ["TABPFN_TOKEN"] = value.strip() + return True + return False + + +@dataclass +class Servable: + """One model, loaded and ready to score a batch of candidate rows.""" + name: str + task: str + family: str + path: Path + classes: np.ndarray + weighted_f1: float = 0.0 + _predict: Callable[[np.ndarray], np.ndarray] | None = field( + default=None, repr=False) + + @property + def cost_hint(self) -> str: + return COST_HINT.get(self.family, "") + + def predict_proba(self, X: np.ndarray) -> np.ndarray: + """Class probabilities for every row of X, in one call.""" + return np.asarray(self._predict(np.asarray(X, dtype=float)), + dtype=float) + + +# ── per-family loading ─────────────────────────────────────────────────────── + +def _load_sklearn(path: Path) -> Servable: + bundle = joblib.load(path) + est = bundle["estimator"] + classes = np.asarray(bundle["classes"], dtype=float) + + def predict(X): + if hasattr(est, "predict_proba"): + return est.predict_proba(X) + # Five conventional classifiers expose no probabilities. A one-hot on + # the predicted label keeps the interface uniform; the expected value + # it produces is then a step function rather than a smooth one, which + # is a property of the model and is stated where it matters. + idx = np.asarray(est.predict(X)).astype(int) + out = np.zeros((len(idx), len(classes))) + out[np.arange(len(idx)), idx] = 1.0 + return out + + return Servable(bundle["model"], bundle["task"], "ml", path, classes, + _predict=predict) + + +def _load_deep(path: Path) -> Servable: + import torch + + from mlate import deep as deep_mod + + bundle = torch.load(path, map_location="cpu", weights_only=False) + classes = np.asarray(bundle["classes"], dtype=float) + net = deep_mod.build(bundle["architecture"], bundle["input_dim"], + len(classes), bundle["params"]) + net.load_state_dict(bundle["state_dict"]) + net.eval() + + def predict(X): + with torch.no_grad(): + logits = net(torch.tensor(X, dtype=torch.float32)) + return torch.softmax(logits, dim=1).numpy() + + return Servable(bundle["architecture"], bundle["task"], "dl", path, + classes, _predict=predict) + + +def _load_foundation(path: Path) -> Servable: + """ + Rebuild the in-context learner and re-supply the corpus it was given. + + `fit` here is not training. It hands the model the stored context and + returns; the checkpoint is unchanged. The call is deferred until the first + prediction so that opening the menu does not pay for every model in it. + """ + bundle = joblib.load(path) + classes = np.asarray(bundle["classes"], dtype=float) + name = bundle["model"] + state: dict = {} + + def fitted(): + if "clf" in state: + return state["clf"] + dev = device() + if name.startswith("TabPFN"): + from tabpfn import TabPFNClassifier + kwargs = dict(device=dev, n_estimators=bundle["n_estimators"], + # 153 features exceeds the pre-training shape, and + # 2,646 context rows exceed what TabPFN will run on + # CPU by default. Both limits are advisory and both + # are lifted by this flag; the second is why a CPU + # host is slow rather than why it refuses. + ignore_pretraining_limits=True, + model_path=bundle.get("checkpoint") + or TABPFN_CHECKPOINT, + random_state=cfg.RANDOM_STATE) + if name.endswith("(thinking)"): + from tabpfn.inference_tuning import ClassifierTuningConfig + kwargs["eval_metric"] = "f1" + kwargs["tuning_config"] = ClassifierTuningConfig( + calibrate_temperature=True, tune_decision_thresholds=True) + os.environ.setdefault("TABPFN_ALLOW_CPU_LARGE_DATASET", "1") + os.environ.setdefault("TABPFN_NO_BROWSER", "1") + _apply_tabpfn_token() + clf = TabPFNClassifier(**kwargs) + elif name == "TabICL": + from tabicl import TabICLClassifier + clf = TabICLClassifier(device=dev, + n_estimators=bundle["n_estimators"], + allow_auto_download=True, + random_state=cfg.RANDOM_STATE) + else: + raise KeyError(name) + clf.fit(bundle["context_X"], bundle["context_y"]) + state["clf"] = clf + return clf + + return Servable(name, bundle["task"], "foundation", path, classes, + _predict=lambda X: fitted().predict_proba(X)) + + +LOADERS = {"ml": _load_sklearn, "dl": _load_deep, + "foundation": _load_foundation} + + +def load(path: Path, family: str) -> Servable: + return LOADERS[family](Path(path)) + + +# ── discovery ──────────────────────────────────────────────────────────────── + +def manifest_scores(models_root: Path, task: str, + protocol: str) -> dict[str, float]: + """ + Benchmarked weighted F1 per model, for ranking the menu. + + Foundation models carry no split protocol - they are not refitted per + split, so the manifest records them as "n/a" - and are scored against the + same held-out test partition as the rest. + """ + p = models_root / "deployment_manifest.json" + if not p.exists(): + return {} + out = {} + for r in json.loads(p.read_text(encoding="utf-8"))["models"]: + if r.get("task") != task: + continue + if r.get("protocol") not in (protocol, "n/a"): + continue + out[r["model"]] = (r.get("metrics") or {}).get("weighted_f1", 0.0) + return out + + +def discover(models_root: Path, task: str, + protocol: str = "random") -> list[Servable]: + """ + Every artefact the application can offer for one target, best first. + + Nothing is loaded here beyond the manifest and each artefact's identity; + the estimators themselves are opened when a model is chosen. Ranking is by + benchmarked weighted F1 so that the head of the menu and the head of the + performance table are the same model, which they were not while the menu + saw only the conventional classifiers. + """ + scores = manifest_scores(models_root, task, protocol) + found: list[Servable] = [] + for family, (sub, per_protocol) in FAMILY_DIRS.items(): + d = models_root / sub / (protocol if per_protocol else "") + if not d.exists(): + continue + for path in sorted(d.glob(f"{task}__*")): + if path.suffix not in (".pkl", ".pt"): + continue + found.append(Stub(path, family, task, scores)) + found.sort(key=lambda s: -s.weighted_f1) + return found + + +@dataclass +class Stub: + """ + A menu entry that has not been opened yet. + + Exists so that listing the models costs a filename parse rather than + unpickling ninety estimators and rebuilding six networks. `open()` returns + the real thing. + """ + path: Path + family: str + task: str + _scores: dict = field(default_factory=dict, repr=False) + name: str = "" + weighted_f1: float = 0.0 + + def __post_init__(self): + if not self.name: + self.name = _display_name(self.path, self.task, self._scores) + self.weighted_f1 = self._scores.get(self.name, 0.0) + + @property + def cost_hint(self) -> str: + return COST_HINT.get(self.family, "") + + def open(self) -> Servable: + s = load(self.path, self.family) + s.weighted_f1 = self.weighted_f1 + return s + + +def _display_name(path: Path, task: str, scores: dict) -> str: + """ + The model's benchmark name, recovered from its filename. + + File names are slugs - "cell_response__k_nn_distance_weighted.pkl" - while + the manifest and the performance table use the printed name, "K-NN + (distance weighted)". Matching on the slug rather than opening the artefact + keeps discovery cheap; an unmatched slug falls back to a readable form of + itself rather than failing, so a newly added model still appears. + """ + stem = path.stem[len(task) + 2:] + for known in scores: + if _slug(known) == stem: + return known + return stem.replace("_", " ").title() + + +def _slug(name: str) -> str: + out = [] + for ch in name.lower(): + out.append(ch if ch.isalnum() else "_") + slug = "".join(out) + while "__" in slug: + slug = slug.replace("__", "_") + return slug.strip("_") diff --git a/mlate/splits.py b/mlate/splits.py new file mode 100644 index 0000000000000000000000000000000000000000..e6ace03cbd9b85d0a6bbb21bfc0c2856067dc749 --- /dev/null +++ b/mlate/splits.py @@ -0,0 +1,249 @@ +""" +Validation splits +================= + +Three protocols, deliberately kept side by side because they answer different +questions and the paper reports all three. + +random stratified hold-out and K-fold. Estimates INTERPOLATION: predicting a + formulation from a study you have already partly seen. Comparable to + the originally submitted evaluation. + +doi StratifiedGroupKFold grouped on DOI, so no study appears in both + train and test. Estimates EXTRAPOLATION to a new study, which is the + condition the web application actually operates in. Primary result. + +tissue leave-one-tissue-out. Supports the cross-tissue claim. + +Why grouping matters here: 89% of rows in a random 20% test set have a +same-study sibling in training, ~45% of label variance sits between studies, +and 68% of rows carry a material+cell-line signature unique to one study. A +model can therefore recognise the study and inherit its rating offset instead +of learning material-outcome relationships. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Iterator + +import numpy as np +import pandas as pd +from sklearn.model_selection import ( + GroupShuffleSplit, StratifiedGroupKFold, StratifiedKFold, train_test_split, +) + +from mlate import config as cfg +from mlate.dataset import modeling_tissue + + +@dataclass(frozen=True) +class Fold: + """One train/test partition, carrying enough context to audit it.""" + name: str + train_idx: np.ndarray + test_idx: np.ndarray + protocol: str + meta: dict = field(default_factory=dict) + + def __len__(self) -> int: + return len(self.test_idx) + + +def _assert_no_group_leak(train_idx, test_idx, groups: np.ndarray, + label: str) -> None: + shared = set(groups[train_idx]) & set(groups[test_idx]) + if shared: + raise AssertionError( + f"{label}: {len(shared)} group(s) appear in both train and test, " + f"e.g. {sorted(shared)[:3]}") + + +def random_folds(y: pd.Series, n_splits: int = cfg.N_FOLDS, + seed: int = cfg.RANDOM_STATE) -> list[Fold]: + """Stratified K-fold ignoring study membership - the interpolation view.""" + skf = StratifiedKFold(n_splits=n_splits, shuffle=True, random_state=seed) + idx = np.arange(len(y)) + return [ + Fold(f"random_fold{i}", idx[tr], idx[te], "random", {"seed": seed}) + for i, (tr, te) in enumerate(skf.split(idx, y)) + ] + + +def random_holdout(y: pd.Series, test_size: float = cfg.TEST_SIZE, + seed: int = cfg.RANDOM_STATE, + stratify: pd.Series | None = None) -> Fold: + """ + Single stratified hold-out, matching the originally submitted design. + + `stratify` defaults to `y`, but callers evaluating more than one target + should pass ONE shared column so every target is scored on the same + scaffold records. The submitted manuscript states this explicitly in 2.7: + "A single shared split was used for both target variables to ensure that + printability and cell-response models were evaluated on the same scaffold + records." + + It is not only a matter of matching the submitted design. WSSQ combines a + printability prediction and a cell-response prediction for the SAME + scaffold; if the two targets are split independently their test sets only + partially overlap, and WSSQ becomes computable on the intersection alone. + """ + idx = np.arange(len(y)) + key = y if stratify is None else stratify + tr, te = train_test_split(idx, test_size=test_size, stratify=key, + random_state=seed, shuffle=True) + return Fold("random_holdout", tr, te, "random", + {"seed": seed, "test_size": test_size, + "stratified_on": getattr(key, "name", "target"), + "shared_across_targets": stratify is not None}) + + +def doi_holdout(y: pd.Series, groups: pd.Series, + test_size: float = cfg.TEST_SIZE, + seed: int = cfg.RANDOM_STATE) -> Fold: + """ + Single study-grouped hold-out: the grouped analogue of `random_holdout`. + + GroupShuffleSplit partitions whole DOIs, so the requested fraction is met + in studies rather than in rows and the realised test share will not land + exactly on `test_size`. That is inherent to grouping - a study is + indivisible - and the realised counts are recorded on the fold so the + Methods can state what was actually held out rather than what was asked + for. + """ + g = groups.to_numpy() + idx = np.arange(len(y)) + gss = GroupShuffleSplit(n_splits=1, test_size=test_size, random_state=seed) + tr, te = next(gss.split(idx, y, groups=g)) + _assert_no_group_leak(tr, te, g, "doi_holdout") + return Fold( + "doi_holdout", idx[tr], idx[te], "doi", + {"seed": seed, "requested_test_size": test_size, + "realised_test_size": round(len(te) / len(idx), 4), + "n_train_studies": len(set(g[tr])), + "n_test_studies": len(set(g[te]))}, + ) + + +def make_holdout(df: pd.DataFrame, y: pd.Series, protocol: str) -> list[Fold]: + """ + The single train/test partition for the hold-out evaluation design. + + Returned as a one-element list so callers can treat the hold-out and + cross-validated designs through the same interface. Under this design the + 10-fold cross-validation happens INSIDE the training partition, for + hyper-parameter selection only, and the returned test partition is scored + exactly once. + """ + # Stratified on Printability for every target, so the two targets share + # one partition. The grouped hold-out is shared automatically, because + # GroupShuffleSplit partitions on studies and never looks at the target. + if protocol == "random": + shared = df[cfg.TARGETS[0]] if cfg.TARGETS[0] in df.columns else None + return [random_holdout(y, stratify=shared)] + if protocol == "doi": + return [doi_holdout(y, df["DOI"])] + raise ValueError(f"no hold-out defined for protocol {protocol!r}") + + +def doi_folds(y: pd.Series, groups: pd.Series, n_splits: int = cfg.N_FOLDS, + seed: int = cfg.RANDOM_STATE) -> list[Fold]: + """ + Study-grouped stratified K-fold. The primary protocol. + + StratifiedGroupKFold balances the class distribution as well as it can + while keeping every DOI wholly inside one fold; perfect stratification is + not achievable under a grouping constraint, which is expected. + """ + g = groups.to_numpy() + sgkf = StratifiedGroupKFold(n_splits=n_splits, shuffle=True, + random_state=seed) + idx = np.arange(len(y)) + folds = [] + for i, (tr, te) in enumerate(sgkf.split(idx, y, groups=g)): + _assert_no_group_leak(tr, te, g, f"doi_fold{i}") + folds.append(Fold( + f"doi_fold{i}", idx[tr], idx[te], "doi", + {"seed": seed, + "n_train_studies": len(set(g[tr])), + "n_test_studies": len(set(g[te]))}, + )) + return folds + + +def tissue_folds(y: pd.Series, tissues: pd.Series, doi: pd.Series, + min_test: int = 30) -> list[Fold]: + """ + Leave-one-tissue-out, with the study boundary enforced as well. + + A study can contribute rows to more than one tissue. Holding out a tissue + without also removing those studies from training would let the model see + the same lab, bioink system and rating style it is about to be tested on, + which is the very effect the grouped protocol exists to remove. Any study + represented in the held-out tissue is therefore dropped from training, and + the count of rows this costs is recorded on the fold. + """ + t = tissues.to_numpy() + g = doi.to_numpy() + idx = np.arange(len(y)) + folds = [] + for tissue in pd.unique(t): + te = idx[t == tissue] + contaminated = set(g[te]) + tr = idx[(t != tissue) & ~np.isin(g, list(contaminated))] + naive = int(((t != tissue).sum()) - len(tr)) + if len(te) < min_test or y.iloc[tr].nunique() < y.nunique(): + continue + _assert_no_group_leak(tr, te, t, f"tissue_{tissue}") + _assert_no_group_leak(tr, te, g, f"tissue_{tissue} (study)") + folds.append(Fold( + f"tissue_{tissue}", tr, te, "tissue", + {"held_out_tissue": tissue, "n_test": len(te), + "train_rows_dropped_for_study_overlap": naive, + "n_train_studies": len(set(g[tr]))}, + )) + return folds + + +def make_folds(df: pd.DataFrame, y: pd.Series, + protocols=("random", "doi", "tissue")) -> list[Fold]: + """Build every requested protocol for one task.""" + folds: list[Fold] = [] + if "random" in protocols: + folds += random_folds(y) + if "doi" in protocols: + folds += doi_folds(y, df["DOI"]) + if "tissue" in protocols: + folds += tissue_folds(y, modeling_tissue(df), df["DOI"]) + return folds + + +def unseen_material_mask(df: pd.DataFrame, fold: Fold, + biomaterials: list[str]) -> np.ndarray: + """ + Which test rows contain a material absent from this fold's training data. + + 63 of 130 materials appear in exactly one study, so a grouped fold can + hold out the only evidence for a material. Predictions for those rows are + uninformed rather than merely unbiased, and the paper reports grouped + performance split on this mask so the two effects are not conflated. + """ + present = (df[biomaterials].fillna(0) != 0).to_numpy() + seen = present[fold.train_idx].any(axis=0) + return present[fold.test_idx][:, ~seen].any(axis=1) + + +def describe(folds: list[Fold], y: pd.Series) -> pd.DataFrame: + """Summary of a fold set, for the supplement and for sanity checks.""" + rows = [] + for f in folds: + rows.append({ + "fold": f.name, + "protocol": f.protocol, + "n_train": len(f.train_idx), + "n_test": len(f.test_idx), + "test_classes": y.iloc[f.test_idx].nunique(), + "train_classes": y.iloc[f.train_idx].nunique(), + **f.meta, + }) + return pd.DataFrame(rows) diff --git a/mlate/style.py b/mlate/style.py new file mode 100644 index 0000000000000000000000000000000000000000..ad3f00e21bb304cba60e222f6f812fd50fc56bd1 --- /dev/null +++ b/mlate/style.py @@ -0,0 +1,388 @@ +""" +MLATE V3 β€” figure design system +=============================== + +One source of truth for the look of every figure in the paper. Import this +before plotting; never set colours, fonts or sizes ad hoc in a figure script. + + from mlate import style as ms + ms.apply() + fig, ax = ms.figure(width="single") + ... + ms.save(fig, "fig1_dataset_growth") + +This module is the project-specific instance of a portable design system kept +as a skill at ~/.claude/skills/scientific-figures/. That copy carries the same +palette, geometry and helpers with no project imports, plus reference/principles.md +- the rules about honest projection, shared coordinate systems and plotting the +criterion that actually made the decision. Port improvements in both directions. + +Design criteria +--------------- +minimal no chartjunk. No frames, no shadows, no gradients, no 3-D. Grid is + a faint single-axis reference, drawn behind the data. Legends have + no box. Only the two spines that carry information are kept. +chic restrained palette, generous whitespace, size and weight used + sparingly to build hierarchy instead of colour and bold. +publication serif type matching the manuscript body, column-accurate widths, + vector PDF plus 600-dpi PNG, every glyph legible at print size. +""" + +from __future__ import annotations + +from pathlib import Path + +import matplotlib as mpl +import matplotlib.pyplot as plt +from matplotlib.colors import LinearSegmentedColormap + +# ───────────────────────────────────────────────────────────────────────────── +# PALETTE +# Extends the navy/teal already used in the submitted figures so the revision +# stays visually continuous with what the reviewers have seen. Ordered by +# perceptual distinctness: the first three read clearly in greyscale. +# ───────────────────────────────────────────────────────────────────────────── +NAVY = "#22405C" +TEAL = "#2E7D6F" +RUST = "#C0663F" +GOLD = "#D9A441" +SLATE = "#7B8794" +PLUM = "#6B5B8A" +SAGE = "#8FA88B" +CLAY = "#A8756A" + +CATEGORICAL = [NAVY, TEAL, RUST, GOLD, PLUM, SAGE, CLAY, SLATE] + +# Eleven biomaterial classes need eleven stable colours. Fixed here so a class +# keeps its colour across Figure 2, Figure 7 and every supplementary panel. +MATERIAL_CLASS_COLORS = { + "natural_polymer": NAVY, + "modified_natural_polymer": TEAL, + "synthetic_polymer": RUST, + "crosslinker": GOLD, + "small_molecule": PLUM, + "bioactive_material": SAGE, + "nanomaterial": CLAY, + "serum_buffer_plasma": SLATE, + "bioceramics": "#4F6D8C", + "enzyme": "#9C8AA5", + "clay": "#BFA980", + "initiator": "#E0A76B", # split out of crosslinker + "ECM_derived": "#3F6B5E", # split out of natural_polymer +} + +# ───────────────────────────────────────────────────────────────────────────── +# ENCODINGS +# Fixed here, not in the figure scripts, so a reader can carry one key across +# every panel of the paper and the supplement. +# ───────────────────────────────────────────────────────────────────────────── + +# Validation protocol. Navy is the interpolation estimate the main text leads +# with; slate is the study-grouped estimate reported alongside it. +PROTOCOL_COLORS = {"random": NAVY, "doi": SLATE, "tissue": RUST} +PROTOCOL_LABELS = {"random": "random split", + "doi": "study-grouped (DOI)", + "tissue": "leave-one-tissue-out"} + +# Broad model group, for figures that put all three benchmarks on one axis. +GROUP_COLORS = {"ml": NAVY, "dl": TEAL, "foundation": RUST, + "baseline": SLATE} +GROUP_LABELS = {"ml": "conventional ML", "dl": "deep learning", + "foundation": "tabular foundation", "baseline": "baseline"} + +# ───────────────────────────────────────────────────────────────────────────── +# ALGORITHM FAMILY β€” the mapping already used in Figures S9-S11 +# +# Moved here from 04_machine_learning/figures_by_protocol.py so that the deep +# and foundation figures can use the same key. A family must keep its colour +# across every panel of the paper; two figures that disagree about what teal +# means are worse than two figures with no colour at all. +# +# Eleven conventional families exhaust the eight palette hues, so three are +# tinted from existing ones rather than pulled from a second palette. The two +# additions at the end cover the models that step 05 introduces. +# ───────────────────────────────────────────────────────────────────────────── +FAMILY_COLORS = { + "baseline": SLATE, + "linear": NAVY, + "discriminant": PLUM, + "naive_bayes": TEAL, + "neighbours": SAGE, + "svm": GOLD, + "tree": CLAY, + "bagging": RUST, + "boosting": "#1C5E72", # deep cyan-navy + "neural": "#8E6FA8", # light plum + "meta": "#5C7A54", # deep sage + "transformer": "#C08A5E", # warm clay-gold, for FT-Transformer/TabNet + "foundation": "#9C4F3F", # deep rust, for TabPFN/TabICL +} + +FAMILY_LABELS = { + "baseline": "baseline", "linear": "linear", + "discriminant": "discriminant", "naive_bayes": "naive Bayes", + "neighbours": "neighbours", "svm": "SVM", "tree": "tree", + "bagging": "bagging", "boosting": "boosting", "neural": "neural", + "meta": "meta-ensemble", "transformer": "transformer", + "foundation": "foundation model", +} + +# Abbreviations used on figure axes, where a full registry name would eat the +# row-label budget. Kept here so every figure shortens a name the same way. +SHORT_NAMES = { + "Logistic Regression": "Logistic Reg.", + "Logistic Regression (balanced)": "Logistic Reg. (bal.)", + "Random Forest (balanced)": "Random Forest (bal.)", + "Quadratic Discriminant": "Quadratic Discrim.", + "Linear Discriminant": "Linear Discrim.", + "Hist Gradient Boosting": "Hist Grad. Boosting", + "Gradient Boosting": "Grad. Boosting", + "k-Nearest Neighbours": "k-NN", + "k-NN (distance weighted)": "k-NN (dist. weighted)", + "Bernoulli Naive Bayes": "Bernoulli NB", + "Gaussian Naive Bayes": "Gaussian NB", + "Soft Voting (RF+XGB+LR)": "Soft Voting", + "Stacking (RF+XGB+LR -> LR)": "Stacking", + "Dummy (majority)": "Dummy - majority", + "Dummy (stratified)": "Dummy - stratified", + "MLP (256-128)": "MLP", + "FT_Transformer": "FT-Transformer", + "TabNet_Lite": "TabNet-Lite", + "NODE_Lite": "NODE-Lite", + "1D_CNN": "1D-CNN", + "TabPFN (thinking)": "TabPFN (thinking)", +} + + +def short(name: str) -> str: + """Figure-axis label for a model, abbreviated consistently.""" + return SHORT_NAMES.get(name, name.replace("_", " ")) + +# Train/test partition, for the overfitting diagnostic. Test is the reported +# quantity and takes the lead hue; train is a muted wash, because a training +# score is a diagnostic and must never read as a result. +SPLIT_COLORS = {"test": NAVY, "train": SLATE} + +# ───────────────────────────────────────────────────────────────────────────── +# METRIC ORDER β€” one canonical sequence for every panel in the paper +# +# Referee 1, comment 5: "the metric ordering in the Figure 6 heatmaps is not +# consistent across panels - the conventional-ML panels end with AUC, MCC, +# Kappa, whereas the DL/zero-shot panels end with Kappa, MCC, AUC. A single +# consistent metric order across all panels would aid comparison." +# +# The order below is that of the submitted manuscript's conventional-ML panels, +# so the fix moves the minority of panels rather than retraining every reader, +# and it groups sensibly: overall accuracy, then the precision/recall/F1 trio, +# then ranking ability, then the two chance-corrected agreement statistics. +# +# Every figure and table script must take its column order from here. Hard- +# coding a list in a figure script is what produced the inconsistency the +# referee found. +# ───────────────────────────────────────────────────────────────────────────── +METRIC_ORDER = ("accuracy", "weighted_precision", "weighted_recall", + "weighted_f1", "roc_auc_weighted_ovr", "mcc", "kappa") + +# The wider panel, for supplementary tables that also carry the imbalance-aware +# and ordinal metrics. The first seven are METRIC_ORDER, unchanged, so a reader +# moving between a main-text figure and a supplementary table reads the same +# sequence and then continues. +METRIC_ORDER_FULL = METRIC_ORDER + ( + "balanced_accuracy", "macro_f1", "quadratic_kappa") + +METRIC_LABELS = { + "accuracy": "Accuracy", + "weighted_precision": "Precision", + "weighted_recall": "Recall", + "weighted_f1": "F1", + "roc_auc_weighted_ovr": "AUC", + "mcc": "MCC", + "kappa": "Kappa", + "balanced_accuracy": "Balanced acc.", + "macro_f1": "Macro F1", + "quadratic_kappa": "Quadratic ΞΊ", +} + + +def metric_columns(full: bool = False) -> list[str]: + """The canonical metric sequence. Never hard-code this in a figure script.""" + return list(METRIC_ORDER_FULL if full else METRIC_ORDER) + + +def metric_headers(full: bool = False) -> list[str]: + """Display labels for `metric_columns`, in the same order.""" + return [METRIC_LABELS[m] for m in metric_columns(full)] + + +# Ordinal target scales: low-to-high, single-hue so magnitude reads directly. +SEQ_NAVY = LinearSegmentedColormap.from_list("mlate_navy", ["#E8EDF2", NAVY]) +SEQ_TEAL = LinearSegmentedColormap.from_list("mlate_teal", ["#E9F1EF", TEAL]) +DIVERGING = LinearSegmentedColormap.from_list("mlate_div", [NAVY, "#F2F0EC", RUST]) + +# Washed tints of the two lead hues, for filled areas that must sit behind a +# stroked outline of the same colour - box faces, confidence bands, the shaded +# half of a paired comparison. Defined here because a figure that needs a pale +# navy should not invent one: 04_machine_learning/figures.py previously carried +# `ms.NAVY_WASH if hasattr(ms, "NAVY_WASH") else "#EAF0F6"`, which is precisely +# the ad-hoc colour this module exists to prevent. +NAVY_WASH = "#EAF0F6" +TEAL_WASH = "#E9F1EF" +RUST_WASH = "#F7ECE7" + +MISSING = "#D9DCE0" # neutral fill for "not reported" +GRIDLINE = "#D6D9DD" +TEXT = "#1A1A1A" +MUTED = "#6E747C" + +# ───────────────────────────────────────────────────────────────────────────── +# GEOMETRY β€” IOP / Biofabrication column widths, in inches +# ───────────────────────────────────────────────────────────────────────────── +WIDTHS = {"single": 3.46, "onehalf": 5.12, "double": 7.09} +DPI = 600 +from mlate import config as cfg + +OUT_DIR = cfg.RESULTS_DIR # overridden per step via save(..., step=) + +# Serif, to match the manuscript body text. Flip SERIF to False for a +# sans-serif build if a journal ever asks for it. +SERIF = True +_FONT_STACK = (["Times New Roman", "Nimbus Roman", "DejaVu Serif"] if SERIF + else ["Arial", "Helvetica", "DejaVu Sans"]) + + +def apply() -> None: + """Install the house style. Call once, before any plotting.""" + mpl.rcParams.update({ + # type + "font.family": "serif" if SERIF else "sans-serif", + ("font.serif" if SERIF else "font.sans-serif"): _FONT_STACK, + "font.size": 8, + "axes.titlesize": 9, + "axes.labelsize": 8, + "xtick.labelsize": 7, + "ytick.labelsize": 7, + "legend.fontsize": 7, + "figure.titlesize": 10, + "mathtext.fontset": "stix", + + # colour + "text.color": TEXT, + "axes.labelcolor": TEXT, + "axes.edgecolor": MUTED, + "xtick.color": MUTED, + "ytick.color": MUTED, + "axes.prop_cycle": mpl.cycler(color=CATEGORICAL), + + # minimal frame - only the informative spines survive + "axes.spines.top": False, + "axes.spines.right": False, + "axes.linewidth": 0.6, + "xtick.major.width": 0.6, + "ytick.major.width": 0.6, + "xtick.major.size": 2.5, + "ytick.major.size": 2.5, + "xtick.direction": "out", + "ytick.direction": "out", + + # faint reference grid, always behind the data + "axes.grid": True, + "axes.axisbelow": True, + "grid.color": GRIDLINE, + "grid.linewidth": 0.5, + "grid.alpha": 0.7, + + # unboxed legend + "legend.frameon": False, + "legend.handlelength": 1.2, + "legend.handletextpad": 0.5, + "legend.columnspacing": 1.2, + "legend.borderaxespad": 0.0, + + # output + "figure.dpi": 150, + "savefig.dpi": DPI, + "figure.facecolor": "white", + "savefig.facecolor": "white", + "savefig.bbox": "tight", + "savefig.pad_inches": 0.02, + "pdf.fonttype": 42, # embed as TrueType, keeps text selectable + "ps.fonttype": 42, + "svg.fonttype": "none", + + "lines.linewidth": 1.2, + "lines.markersize": 3.5, + "patch.linewidth": 0.0, + }) + + +def figure(width: str = "single", height: float | None = None, + nrows: int = 1, ncols: int = 1, **kw): + """ + Create a correctly-sized figure. + + width 'single' | 'onehalf' | 'double', or a number in inches + height inches; defaults to a 0.68 aspect per axes row + """ + w = WIDTHS.get(width, width) if isinstance(width, str) else width + h = height if height is not None else w * 0.68 * nrows + return plt.subplots(nrows, ncols, figsize=(w, h), **kw) + + +def grid_axis(ax, axis: str = "y") -> None: + """Restrict the grid to the single axis that carries magnitude.""" + ax.grid(False) + ax.grid(True, axis=axis) + + +def despine(ax, keep=("left", "bottom")) -> None: + for side in ("top", "right", "left", "bottom"): + ax.spines[side].set_visible(side in keep) + + +def label_bars(ax, bars, values, fmt="{:.0f}", pad=0.01, + horizontal=False, size=6.5) -> None: + """Direct-label bars so the reader never has to trace back to an axis.""" + span = (ax.get_xlim()[1] if horizontal else ax.get_ylim()[1]) or 1 + for bar, v in zip(bars, values): + if horizontal: + ax.text(bar.get_width() + pad * span, + bar.get_y() + bar.get_height() / 2, + fmt.format(v), va="center", ha="left", + size=size, color=MUTED) + else: + ax.text(bar.get_x() + bar.get_width() / 2, + bar.get_height() + pad * span, + fmt.format(v), ha="center", va="bottom", + size=size, color=MUTED) + + +def halo(width: float = 2.0, colour: str = "white") -> list: + """ + A white outline for text sitting on top of dense marks. + + Cluster labels land in the middle of a scatter, where plain text on either + a light or a dark patch of points is unreadable. A stroke behind the glyph + keeps the label legible without a filled box, which would hide the data it + is annotating. + """ + from matplotlib import patheffects + return [patheffects.withStroke(linewidth=width, foreground=colour)] + + +def panel_tag(ax, tag: str, dx: float = -0.14, dy: float = 1.04) -> None: + """Panel letter, upper-left, unbolded - hierarchy comes from position.""" + ax.text(dx, dy, tag, transform=ax.transAxes, + size=9.5, weight="semibold", va="top", ha="left", color=TEXT) + + +def save(fig, name: str, step: str = "01_data", + formats=("png", "pdf")) -> list[Path]: + """Write vector + raster copies under results//figures/.""" + out = cfg.step_dir(step, "figures") + written = [] + for ext in formats: + path = out / f"{name}.{ext}" + fig.savefig(path, format=ext) + written.append(path) + plt.close(fig) + return written diff --git a/mlate/wssq.py b/mlate/wssq.py new file mode 100644 index 0000000000000000000000000000000000000000..bad63e4a101822f1b23ca4c4c1d8c5ab44816f24 --- /dev/null +++ b/mlate/wssq.py @@ -0,0 +1,166 @@ +""" +Weighted Synergistic Scaffold Quality (WSSQ) +============================================ + +The single score the optimiser maximises, combining Printability and Cell +Response. This module is the canonical implementation: the web application, +the Bayesian optimiser and the sensitivity analysis in `07_wssq/` all import +from here, so there is exactly one definition of the metric in the project. + +The arithmetic is transcribed unchanged from the deployed V3 application +(`app/legacy/wssq.py`). Nothing here alters a published score; what is added is +the documentation of *why* the form is what it is, which the referees asked for, +and the constants named so the sensitivity analysis can vary them. + +The construction +---------------- +Both targets are first mapped onto [0, 1]: + + norm_p = p / 3 Printability 0-3 + norm_c = (c - 1) / 4 Cell Response 1-5 + +They are then combined by two weighted means, with weights wp and wc summing +to one: + + HWM = 1 / (wp/norm_p + wc/norm_c) weighted harmonic mean + WMC = norm_p**wp * norm_c**wc weighted geometric mean + WSSQ = 100 * (blend * HWM + (1 - blend) * WMC) + +Why two conjunctive means rather than one arithmetic mean +--------------------------------------------------------- +An arithmetic mean is *compensatory*: excellent printability offsets dead +cells, and a scaffold that fails one objective outright can still score well. +That is the wrong behaviour for a screening score whose purpose is to surface +candidates worth attempting in a laboratory. + +Both means used here are *conjunctive* - each goes to zero when either +component goes to zero, so neither objective can be traded away entirely. They +differ in how hard they punish imbalance, and the weighted mean inequality + + HWM <= WMC <= arithmetic mean + +orders them: the harmonic mean is the more severe of the two, and both are +bounded above by the arithmetic mean the metric deliberately avoids. + +`blend` therefore is not an arbitrary mixing constant - it selects severity +inside a bounded family. blend = 1 is the most conservative scoring the family +permits, blend = 0 the most permissive, and because WSSQ is linear in `blend` +the whole family lies between the two endpoints. The shipped default of 0.5 +sits at the midpoint. `07_wssq/sensitivity.py` reports how much the ranking of +candidate formulations actually moves across that range. + +Two boundary rules +------------------ +`p == 0` scores zero. An ink that does not extrude is not a scaffold, and no +biological performance redeems it. + +`c <= 1` falls back to printability alone, scaled to [0, 100]. Cell Response 1 +means *no cells were cultured*, not *cells did badly*; scoring an acellular +scaffold as though it had failed a biological test would be wrong, and the +platform is explicitly intended to serve acellular 3D-printed scaffolds as well +as bioprinted ones. Note the consequence, quantified in the sensitivity +analysis: for any candidate on this branch the weights cancel out entirely, so +its score is weight-invariant by construction. +""" + +from __future__ import annotations + +import numpy as np + +# Shipped defaults - but the three are not on the same footing, and the +# distinction is what the two referee comments turn on. +# +# print_weight / cell_weight USER-CONTROLLED. The application exposes cell +# weight as a sidebar slider (0-100% in steps of 5, default 70), with +# print weight taking the remainder. 0.3/0.7 is where the slider starts, +# not a fixed modelling choice, so the sensitivity analysis over these is +# a map of the tool across its own input range rather than a defence of +# one setting. +# +# blend FIXED. Not exposed anywhere in the interface; +# every user of the platform scores at 0.5. It is therefore the only +# constant in the metric that has to be justified on its own, which is +# precisely what referee 2 asked about equation 6. +DEFAULT_PRINT_WEIGHT = 0.3 +DEFAULT_CELL_WEIGHT = 0.7 +DEFAULT_BLEND = 0.5 + +# The slider's actual positions, so the analysis samples what users can +# really select rather than a continuum they cannot reach. +SLIDER_CELL_WEIGHTS = tuple(w / 100 for w in range(0, 101, 5)) + +PRINTABILITY_RANGE = (0.0, 3.0) +CELL_RESPONSE_RANGE = (1.0, 5.0) + + +def compute_wssq( + printability, + cell_response, + print_weight: float = DEFAULT_PRINT_WEIGHT, + cell_weight: float = DEFAULT_CELL_WEIGHT, + blend: float = DEFAULT_BLEND, +): + """Canonical MLATE V3 WSSQ implementation shared by the deployed app.""" + p = np.asarray(printability, dtype=float) + c = np.asarray(cell_response, dtype=float) + if np.any(~np.isfinite(p)) or np.any(~np.isfinite(c)): + raise ValueError("WSSQ inputs must be finite") + if np.any((p < 0) | (p > 3)) or np.any((c < 1) | (c > 5)): + raise ValueError("WSSQ inputs are outside their label domains") + if print_weight < 0 or cell_weight < 0 or print_weight + cell_weight <= 0: + raise ValueError("WSSQ weights must be non-negative and not both zero") + if not 0 <= blend <= 1: + raise ValueError("blend must be within [0, 1]") + total = print_weight + cell_weight + wp, wc = print_weight / total, cell_weight / total + norm_p = p / 3.0 + norm_c = (c - 1.0) / 4.0 + epsilon = np.finfo(float).eps + harmonic = 1.0 / (wp / np.maximum(norm_p, epsilon) + + wc / np.maximum(norm_c, epsilon)) + multiplicative = (np.maximum(norm_p, epsilon) ** wp + * np.maximum(norm_c, epsilon) ** wc) + score = 100.0 * (blend * harmonic + (1.0 - blend) * multiplicative) + score = np.where(p == 0, 0.0, np.where(c <= 1, 100.0 * norm_p, score)) + score = np.clip(score, 0.0, 100.0) + return float(score) if score.ndim == 0 else score + + +def components(printability, cell_response, + print_weight: float = DEFAULT_PRINT_WEIGHT, + cell_weight: float = DEFAULT_CELL_WEIGHT): + """ + The two means before blending, on the 0-100 scale, plus the arithmetic + mean the metric deliberately does not use. + + Exposed so the sensitivity analysis can verify the HWM <= WMC <= AM + ordering on the actual data rather than asserting it from the inequality. + Boundary rules are NOT applied here: this returns the raw aggregators. + """ + p = np.asarray(printability, dtype=float) + c = np.asarray(cell_response, dtype=float) + total = print_weight + cell_weight + wp, wc = print_weight / total, cell_weight / total + norm_p, norm_c = p / 3.0, (c - 1.0) / 4.0 + eps = np.finfo(float).eps + np_, nc_ = np.maximum(norm_p, eps), np.maximum(norm_c, eps) + return { + "harmonic": 100.0 / (wp / np_ + wc / nc_), + "multiplicative": 100.0 * np_ ** wp * nc_ ** wc, + "arithmetic": 100.0 * (wp * np_ + wc * nc_), + } + + +def is_weight_invariant(printability, cell_response) -> np.ndarray: + """ + Candidates whose score cannot move when the weights or the blend change. + + Both boundary rules bypass the weighted means entirely: `p == 0` returns a + constant zero, and `c <= 1` returns a function of printability alone. For + those rows every weighting produces the same number, so they contribute no + information to a weight sensitivity analysis and would dilute any rank + correlation computed over the whole corpus if left unflagged. + """ + p = np.asarray(printability, dtype=float) + c = np.asarray(cell_response, dtype=float) + return (p == 0) | (c <= 1) diff --git a/model_performance.py b/model_performance.py new file mode 100644 index 0000000000000000000000000000000000000000..4276e69dd9d565d38240e6b1beae622d550fa203 --- /dev/null +++ b/model_performance.py @@ -0,0 +1,137 @@ +""" +Benchmarked test-set performance shown in the interface. + +GENERATED by 06_webapp/build_app_data.py - do not edit by hand. +Regenerate after any change to the dataset or the benchmark tables. +""" + +# Random-split hold-out, test partition, weighted-F1 selection. +# n = 530 test records for both targets. + +PERFORMANCE_PROTOCOL = 'random' + +PERFORMANCE_GUIDE = { + 'printability': """ +| Model | Type | In menu | Accuracy | Weighted F1 | Macro F1 | MCC | ROC-AUC | +|---|---|---|---|---|---|---|---| +| TabICL | Foundation | yes | 0.806 | 0.806 | 0.771 | 0.702 | 0.930 | +| Bagged Trees | ML | yes | 0.794 | 0.792 | 0.763 | 0.677 | 0.915 | +| CatBoost | ML | yes | 0.789 | 0.786 | 0.754 | 0.669 | 0.918 | +| Hist Gradient Boosting | ML | yes | 0.787 | 0.784 | 0.758 | 0.667 | 0.917 | +| TabPFN (thinking) | Foundation | yes | 0.779 | 0.779 | 0.738 | 0.659 | 0.918 | +| Stacking (RF+XGB+LR -> LR) | ML | yes | 0.775 | 0.773 | 0.730 | 0.649 | 0.916 | +| TabPFN | Foundation | yes | 0.770 | 0.770 | 0.719 | 0.644 | 0.918 | +| Gradient Boosting | ML | yes | 0.766 | 0.763 | 0.713 | 0.633 | 0.914 | +| LightGBM | ML | yes | 0.764 | 0.762 | 0.712 | 0.633 | 0.914 | +| XGBoost | ML | yes | 0.762 | 0.760 | 0.709 | 0.631 | 0.914 | +| Extra Trees | ML | yes | 0.760 | 0.760 | 0.713 | 0.628 | 0.910 | +| Soft Voting (RF+XGB+LR) | ML | yes | 0.760 | 0.756 | 0.703 | 0.625 | 0.900 | +""", + 'cell_response': """ +| Model | Type | In menu | Accuracy | Weighted F1 | Macro F1 | MCC | ROC-AUC | +|---|---|---|---|---|---|---|---| +| Stacking (RF+XGB+LR -> LR) | ML | yes | 0.785 | 0.773 | 0.554 | 0.641 | 0.949 | +| TabPFN (thinking) | Foundation | yes | 0.774 | 0.773 | 0.555 | 0.625 | 0.960 | +| Bagged Trees | ML | yes | 0.777 | 0.773 | 0.555 | 0.629 | 0.953 | +| TabPFN | Foundation | yes | 0.781 | 0.772 | 0.534 | 0.636 | 0.958 | +| Hist Gradient Boosting | ML | yes | 0.770 | 0.767 | 0.541 | 0.617 | 0.957 | +| Random Forest (balanced) | ML | yes | 0.764 | 0.766 | 0.560 | 0.619 | 0.946 | +| TabICL | Foundation | yes | 0.770 | 0.766 | 0.546 | 0.615 | 0.959 | +| FT_Transformer | DL | yes | 0.758 | 0.754 | 0.503 | 0.600 | 0.951 | +| ResNet | DL | yes | 0.762 | 0.753 | 0.530 | 0.600 | 0.950 | +| Quadratic Discriminant | ML | yes | 0.751 | 0.749 | 0.513 | 0.586 | 0.935 | +| Soft Voting (RF+XGB+LR) | ML | yes | 0.758 | 0.744 | 0.504 | 0.591 | 0.950 | +| Bernoulli Naive Bayes | ML | yes | 0.745 | 0.743 | 0.517 | 0.577 | 0.927 | +""", +} + +PERFORMANCE_GUIDE_FULL = { + 'printability': """ +| Model | Type | In menu | Accuracy | Weighted F1 | Macro F1 | MCC | ROC-AUC | +|---|---|---|---|---|---|---|---| +| TabICL | Foundation | yes | 0.806 | 0.806 | 0.771 | 0.702 | 0.930 | +| Bagged Trees | ML | yes | 0.794 | 0.792 | 0.763 | 0.677 | 0.915 | +| CatBoost | ML | yes | 0.789 | 0.786 | 0.754 | 0.669 | 0.918 | +| Hist Gradient Boosting | ML | yes | 0.787 | 0.784 | 0.758 | 0.667 | 0.917 | +| TabPFN (thinking) | Foundation | yes | 0.779 | 0.779 | 0.738 | 0.659 | 0.918 | +| Stacking (RF+XGB+LR -> LR) | ML | yes | 0.775 | 0.773 | 0.730 | 0.649 | 0.916 | +| TabPFN | Foundation | yes | 0.770 | 0.770 | 0.719 | 0.644 | 0.918 | +| Gradient Boosting | ML | yes | 0.766 | 0.763 | 0.713 | 0.633 | 0.914 | +| LightGBM | ML | yes | 0.764 | 0.762 | 0.712 | 0.633 | 0.914 | +| XGBoost | ML | yes | 0.762 | 0.760 | 0.709 | 0.631 | 0.914 | +| Extra Trees | ML | yes | 0.760 | 0.760 | 0.713 | 0.628 | 0.910 | +| Soft Voting (RF+XGB+LR) | ML | yes | 0.760 | 0.756 | 0.703 | 0.625 | 0.900 | +| k-NN (distance weighted) | ML | yes | 0.749 | 0.749 | 0.714 | 0.613 | 0.850 | +| MLP (256-128) | ML | yes | 0.742 | 0.742 | 0.706 | 0.600 | 0.876 | +| ResNet | DL | yes | 0.740 | 0.741 | 0.690 | 0.604 | 0.876 | +| k-Nearest Neighbours | ML | yes | 0.736 | 0.735 | 0.707 | 0.591 | 0.797 | +| SVM (polynomial) | ML | yes | 0.730 | 0.732 | 0.670 | 0.590 | 0.881 | +| Random Forest (balanced) | ML | yes | 0.721 | 0.729 | 0.675 | 0.594 | 0.894 | +| FT_Transformer | DL | yes | 0.726 | 0.728 | 0.678 | 0.583 | 0.869 | +| TabNet_Lite | DL | yes | 0.721 | 0.721 | 0.655 | 0.573 | 0.865 | +| Decision Tree | ML | yes | 0.713 | 0.712 | 0.653 | 0.556 | 0.887 | +| NODE_Lite | DL | yes | 0.685 | 0.697 | 0.622 | 0.543 | 0.863 | +| MLP | DL | yes | 0.685 | 0.694 | 0.631 | 0.538 | 0.867 | +| SVM (RBF) | ML | yes | 0.691 | 0.691 | 0.620 | 0.529 | 0.863 | +| Logistic Regression | ML | yes | 0.672 | 0.665 | 0.574 | 0.492 | 0.829 | +| SGD (hinge) | ML | yes | 0.674 | 0.661 | 0.572 | 0.492 | n/a | +| Linear Discriminant | ML | yes | 0.675 | 0.660 | 0.559 | 0.485 | 0.834 | +| 1D_CNN | DL | yes | 0.657 | 0.654 | 0.558 | 0.479 | 0.850 | +| Passive Aggressive | ML | yes | 0.645 | 0.650 | 0.559 | 0.474 | n/a | +| Extra Tree | ML | yes | 0.658 | 0.649 | 0.556 | 0.468 | 0.863 | +| Linear SVM | ML | yes | 0.634 | 0.646 | 0.551 | 0.469 | n/a | +| Bernoulli Naive Bayes | ML | yes | 0.641 | 0.642 | 0.536 | 0.463 | 0.819 | +| Logistic Regression (balanced) | ML | yes | 0.619 | 0.636 | 0.552 | 0.460 | 0.816 | +| AdaBoost | ML | yes | 0.638 | 0.628 | 0.514 | 0.434 | 0.769 | +| Quadratic Discriminant | ML | yes | 0.619 | 0.620 | 0.513 | 0.439 | 0.816 | +| Ridge Classifier | ML | yes | 0.649 | 0.614 | 0.492 | 0.429 | n/a | +| Random Forest | ML | yes | 0.632 | 0.593 | 0.435 | 0.423 | 0.822 | +| Nearest Centroid | ML | yes | 0.464 | 0.499 | 0.404 | 0.285 | 0.810 | +| Perceptron | ML | yes | 0.536 | 0.439 | 0.320 | 0.199 | n/a | +| Gaussian Naive Bayes | ML | yes | 0.279 | 0.305 | 0.276 | 0.181 | 0.706 | +""", + 'cell_response': """ +| Model | Type | In menu | Accuracy | Weighted F1 | Macro F1 | MCC | ROC-AUC | +|---|---|---|---|---|---|---|---| +| Stacking (RF+XGB+LR -> LR) | ML | yes | 0.785 | 0.773 | 0.554 | 0.641 | 0.949 | +| TabPFN (thinking) | Foundation | yes | 0.774 | 0.773 | 0.555 | 0.625 | 0.960 | +| Bagged Trees | ML | yes | 0.777 | 0.773 | 0.555 | 0.629 | 0.953 | +| TabPFN | Foundation | yes | 0.781 | 0.772 | 0.534 | 0.636 | 0.958 | +| Hist Gradient Boosting | ML | yes | 0.770 | 0.767 | 0.541 | 0.617 | 0.957 | +| Random Forest (balanced) | ML | yes | 0.764 | 0.766 | 0.560 | 0.619 | 0.946 | +| TabICL | Foundation | yes | 0.770 | 0.766 | 0.546 | 0.615 | 0.959 | +| FT_Transformer | DL | yes | 0.758 | 0.754 | 0.503 | 0.600 | 0.951 | +| ResNet | DL | yes | 0.762 | 0.753 | 0.530 | 0.600 | 0.950 | +| Quadratic Discriminant | ML | yes | 0.751 | 0.749 | 0.513 | 0.586 | 0.935 | +| Soft Voting (RF+XGB+LR) | ML | yes | 0.758 | 0.744 | 0.504 | 0.591 | 0.950 | +| Bernoulli Naive Bayes | ML | yes | 0.745 | 0.743 | 0.517 | 0.577 | 0.927 | +| Decision Tree | ML | yes | 0.747 | 0.734 | 0.479 | 0.574 | 0.935 | +| k-NN (distance weighted) | ML | yes | 0.742 | 0.734 | 0.524 | 0.565 | 0.835 | +| Extra Trees | ML | yes | 0.745 | 0.734 | 0.497 | 0.567 | 0.952 | +| CatBoost | ML | yes | 0.747 | 0.729 | 0.455 | 0.575 | 0.951 | +| Logistic Regression (balanced) | ML | yes | 0.725 | 0.729 | 0.467 | 0.550 | 0.935 | +| LightGBM | ML | yes | 0.743 | 0.727 | 0.431 | 0.570 | 0.947 | +| XGBoost | ML | yes | 0.740 | 0.724 | 0.437 | 0.563 | 0.948 | +| k-Nearest Neighbours | ML | yes | 0.730 | 0.722 | 0.495 | 0.545 | 0.820 | +| AdaBoost | ML | yes | 0.734 | 0.712 | 0.420 | 0.556 | 0.910 | +| Extra Tree | ML | yes | 0.740 | 0.712 | 0.385 | 0.564 | 0.933 | +| MLP | DL | yes | 0.723 | 0.709 | 0.447 | 0.531 | 0.933 | +| TabNet_Lite | DL | yes | 0.715 | 0.709 | 0.472 | 0.522 | 0.905 | +| Linear SVM | ML | yes | 0.721 | 0.703 | 0.453 | 0.520 | n/a | +| MLP (256-128) | ML | yes | 0.730 | 0.703 | 0.427 | 0.533 | 0.924 | +| Random Forest | ML | yes | 0.732 | 0.700 | 0.364 | 0.556 | 0.905 | +| Passive Aggressive | ML | yes | 0.715 | 0.696 | 0.435 | 0.509 | n/a | +| NODE_Lite | DL | yes | 0.719 | 0.690 | 0.421 | 0.510 | 0.926 | +| 1D_CNN | DL | yes | 0.721 | 0.689 | 0.403 | 0.517 | 0.918 | +| Linear Discriminant | ML | yes | 0.711 | 0.686 | 0.434 | 0.494 | 0.925 | +| SGD (hinge) | ML | yes | 0.717 | 0.686 | 0.406 | 0.506 | n/a | +| SVM (polynomial) | ML | yes | 0.715 | 0.685 | 0.393 | 0.506 | 0.923 | +| Logistic Regression | ML | yes | 0.715 | 0.684 | 0.405 | 0.502 | 0.925 | +| SVM (RBF) | ML | yes | 0.706 | 0.681 | 0.398 | 0.491 | 0.922 | +| Nearest Centroid | ML | yes | 0.696 | 0.679 | 0.440 | 0.499 | 0.925 | +| Gradient Boosting | ML | yes | 0.736 | 0.667 | 0.308 | 0.590 | 0.897 | +| Perceptron | ML | yes | 0.691 | 0.655 | 0.385 | 0.453 | n/a | +| Ridge Classifier | ML | yes | 0.691 | 0.648 | 0.374 | 0.447 | n/a | +| Gaussian Naive Bayes | ML | yes | 0.270 | 0.311 | 0.260 | 0.196 | 0.808 | +""", +} diff --git a/requirements.txt b/requirements.txt index 28cb376cf5c29622c3d854fd25609f9bb215441d..9a3e865531b460d1fc70d42730adfac4d4e505c1 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,17 +1,40 @@ -joblib==1.5.3 -optuna==4.8.0 -pandas==2.3.3 -numpy==2.0.2 -scipy==1.15.3 -scikit-learn==1.6.1 -pyarrow==20.0.0 -numexpr==2.14.1 -bottleneck==1.6.0 -openai -google-genai -category_encoders==2.9.0 -xgboost==3.2.0 -lightgbm==4.6.0 - ---extra-index-url https://download.pytorch.org/whl/cpu -torch==2.7.1+cpu \ No newline at end of file +# Hugging Face Space runtime for the MLATE V3 application. +# +# Deliberately smaller than the analysis environment at the repository root. +# A Space serves predictions from artefacts that were already fitted; it never +# trains, tunes or clusters, so the training-time dependencies are omitted. +# +# All three model families are served, which is why torch is here. It is the +# largest item in the image by a wide margin, and it is pulled from PyTorch's +# CPU index below: the default wheel carries the CUDA runtime, which a Space +# on CPU hardware cannot use and which multiplies the image size several times +# over. A Space on GPU hardware should drop the two index lines and take the +# default wheel instead. +# +# Cost of the foundation models, measured on this corpus with a batch of 32 +# candidates: about 1 s per pass on a GPU and about 17 s on CPU, against 80 ms +# for the best conventional classifier. A 150-trial search is therefore a few +# seconds, roughly a minute, or roughly three minutes depending on which family +# is chosen and what the host has. The application states this beside the menu. +--extra-index-url https://download.pytorch.org/whl/cpu + +streamlit==1.58.0 +pandas==2.3.3 +numpy==2.4.4 +scipy==1.17.1 +scikit-learn==1.8.0 +category_encoders==2.9.0 +xgboost==3.2.0 +lightgbm==4.6.0 +catboost==1.2.10 +optuna==4.8.0 +joblib==1.5.3 +pyarrow==24.0.0 + +# Deep networks and in-context foundation models. The "+cpu" local version +# exists only on the PyTorch index above, which is what forces pip to resolve +# it there: the plain "torch==2.11.0" on PyPI is the CUDA build on Linux and +# pulls the whole NVIDIA runtime, several gigabytes a CPU Space cannot use. +torch==2.11.0+cpu +tabicl==2.1.1 +tabpfn==7.1.1