|
|
--- |
|
|
license: bsd-3-clause |
|
|
language: |
|
|
- en |
|
|
tags: |
|
|
- python |
|
|
- colour |
|
|
- color |
|
|
- colour-science |
|
|
- color-science |
|
|
- colour-spaces |
|
|
- color-spaces |
|
|
- colourspace |
|
|
- colorspace |
|
|
pipeline_tag: tabular-regression |
|
|
library_name: onnxruntime |
|
|
metrics: |
|
|
- mae |
|
|
model-index: |
|
|
- name: from_xyY (CIE xyY to Munsell) |
|
|
results: |
|
|
- task: |
|
|
type: tabular-regression |
|
|
name: CIE xyY to Munsell Specification |
|
|
dataset: |
|
|
name: CIE xyY to Munsell Specification |
|
|
type: munsell-renotation |
|
|
metrics: |
|
|
- type: delta-e |
|
|
value: 0.51 |
|
|
name: Delta-E CIE2000 |
|
|
- type: inference_time_ms |
|
|
value: 0.061 |
|
|
name: Inference Time (ms/sample) |
|
|
- name: to_xyY (Munsell to CIE xyY) |
|
|
results: |
|
|
- task: |
|
|
type: tabular-regression |
|
|
name: Munsell Specification to CIE xyY |
|
|
dataset: |
|
|
name: Munsell Specification to CIE xyY |
|
|
type: munsell-renotation |
|
|
metrics: |
|
|
- type: delta-e |
|
|
value: 0.48 |
|
|
name: Delta-E CIE2000 |
|
|
- type: inference_time_ms |
|
|
value: 0.066 |
|
|
name: Inference Time (ms/sample) |
|
|
--- |
|
|
|
|
|
# Learning Munsell - Machine Learning for Munsell Color Conversions |
|
|
|
|
|
A project implementing machine learning-based methods for bidirectional conversion between CIE xyY colourspace values and Munsell specifications. |
|
|
|
|
|
**Two Conversion Directions:** |
|
|
|
|
|
- **from_xyY**: CIE xyY to Munsell specification |
|
|
- **to_xyY**: Munsell specification to CIE xyY |
|
|
|
|
|
## Project Overview |
|
|
|
|
|
### Objective |
|
|
|
|
|
Provide 100-1000x speedup for batch Munsell conversions compared to colour-science routines while maintaining high accuracy. |
|
|
|
|
|
### Results |
|
|
|
|
|
**from_xyY** (CIE xyY to Munsell) — 30 models evaluated on all 2,734 REAL Munsell colors: |
|
|
|
|
|
| Model | Delta-E | Speed (ms) | |
|
|
|------------------------------------------------------|------------|------------| |
|
|
| Colour Library (Baseline) | 0.00 | 116.3 | |
|
|
| **Multi-MLP + Multi-Error Predictor** | **0.51** | 0.061 | |
|
|
| Multi-ResNet + Multi-Error Predictor (Large Dataset) | 0.52 | 0.096 | |
|
|
| Transformer + Error Predictor (Large Dataset) | 0.52 | 0.163 | |
|
|
| Multi-Head + Multi-Error Predictor (Large Dataset) | 0.53 | 0.046 | |
|
|
| Multi-MLP (Class. Code) + Multi-Error Predictor | 0.53 | 0.049 | |
|
|
| MLP + Error Predictor | 0.53 | 0.036 | |
|
|
| Multi-Head + Multi-Error Predictor | 0.54 | 0.057 | |
|
|
| Multi-ResNet (Large Dataset) | 0.56 | 0.047 | |
|
|
| Multi-Head (Large Dataset) | 0.57 | 0.012 | |
|
|
| FT-Transformer | 0.70 | 0.067 | |
|
|
| Unified MLP | 0.71 | 0.074 | |
|
|
| Transformer (Large Dataset) | 0.73 | 0.120 | |
|
|
| Mixture of Experts | 0.74 | 0.021 | |
|
|
| Multi-MLP | 0.91 | 0.027 | |
|
|
| Multi-MLP (Classification Code) | 0.92 | 0.026 | |
|
|
| MLP + Self-Attention | 0.88 | 0.185 | |
|
|
| Deep + Wide | 1.18 | 0.078 | |
|
|
| MLP (Base Only) | 1.30 | **0.009** | |
|
|
| Multi-MLP (Hue Angle sin/cos) | 1.78 | 0.021 | |
|
|
|
|
|
- **Best Accuracy**: Multi-MLP + Multi-Error Predictor — Delta-E 0.51, 1,905x faster |
|
|
- **Fastest**: MLP Base Only (0.009 ms/sample) — 13,365x faster than Colour library |
|
|
- **Best Balance**: MLP + Error Predictor — 3,196x faster with Delta-E 0.53 |
|
|
|
|
|
**to_xyY** (Munsell to CIE xyY) — 6 models evaluated on all 2,734 REAL Munsell colors: |
|
|
|
|
|
| Model | Delta-E | Speed (ms) | |
|
|
|---------------------------------------|------------|------------| |
|
|
| Colour Library (Baseline) | 0.00 | 1.40 | |
|
|
| **Multi-MLP + Multi-Error Predictor** | **0.48** | 0.066 | |
|
|
| Multi-Head + Multi-Error Predictor | 0.51 | 0.054 | |
|
|
| Simple MLP | 0.52 | 0.002 | |
|
|
| Multi-MLP | 0.57 | 0.028 | |
|
|
| Multi-Head | 0.60 | 0.013 | |
|
|
| Multi-MLP + Error Predictor | 0.61 | 0.060 | |
|
|
|
|
|
- **Best Accuracy**: Multi-MLP + Multi-Error Predictor — Delta-E 0.48, 21x faster |
|
|
- **Fastest**: Simple MLP (0.002 ms/sample) — 669x faster than Colour library |
|
|
|
|
|
### Approach |
|
|
|
|
|
- **30 models** tested for from_xyY (MLP, Multi-Head, Multi-MLP, Multi-ResNet, Transformers, FT-Transformer, Mixture of Experts) |
|
|
- **6 models** tested for to_xyY (Simple MLP, Multi-Head, Multi-MLP with error predictors) |
|
|
- **Two-stage models** (base + error predictor) proved most effective |
|
|
- **Best model**: Multi-MLP + Multi-Error Predictor with Delta-E 0.51 |
|
|
- **Training data**: ~1.4M samples from dense xyY grid with boundary refinement and forward Munsell sampling |
|
|
- **Deployment**: ONNX format with ONNX Runtime |
|
|
|
|
|
For detailed architecture comparisons, model benchmarks, training pipeline details, and experimental results, see [docs/learning_munsell.md](docs/learning_munsell.md). |
|
|
|
|
|
## Installation |
|
|
|
|
|
**Dependencies (Runtime)**: |
|
|
|
|
|
- numpy >= 2.0 |
|
|
- onnxruntime >= 1.16 |
|
|
|
|
|
**Dependencies (Training)**: |
|
|
|
|
|
- torch >= 2.0 |
|
|
- scikit-learn >= 1.3 |
|
|
- matplotlib >= 3.9 |
|
|
- mlflow >= 2.10 |
|
|
- optuna >= 3.0 |
|
|
- colour-science >= 0.4.7 |
|
|
- click >= 8.0 |
|
|
- onnx >= 1.15 |
|
|
- onnxscript >= 0.5.6 |
|
|
- tqdm >= 4.66 |
|
|
- jax >= 0.4.20 |
|
|
- jaxlib >= 0.4.20 |
|
|
- flax >= 0.10.7 |
|
|
- optax >= 0.2.6 |
|
|
- scipy >= 1.12 |
|
|
- tensorboard >= 2.20 |
|
|
|
|
|
From the project root: |
|
|
|
|
|
```bash |
|
|
cd learning-munsell |
|
|
|
|
|
# Install all dependencies (creates virtual environment automatically) |
|
|
uv sync |
|
|
``` |
|
|
|
|
|
## Usage |
|
|
|
|
|
### Generate Training Data |
|
|
|
|
|
```bash |
|
|
uv run python learning_munsell/data_generation/generate_training_data.py |
|
|
``` |
|
|
|
|
|
**Note**: This step is computationally expensive (uses iterative algorithm for ground truth). |
|
|
|
|
|
### Train Models |
|
|
|
|
|
**xyY to Munsell (from_xyY)** |
|
|
|
|
|
Best performing model (Multi-MLP + Multi-Error Predictor): |
|
|
|
|
|
```bash |
|
|
# Train base Multi-MLP |
|
|
uv run python learning_munsell/training/from_xyY/train_multi_mlp.py |
|
|
|
|
|
# Train multi-error predictor |
|
|
uv run python learning_munsell/training/from_xyY/train_multi_mlp_multi_error_predictor.py |
|
|
``` |
|
|
|
|
|
Alternative architectures: |
|
|
|
|
|
```bash |
|
|
uv run python learning_munsell/training/from_xyY/train_multi_resnet_large.py |
|
|
uv run python learning_munsell/training/from_xyY/train_multi_head_large.py |
|
|
uv run python learning_munsell/training/from_xyY/train_ft_transformer.py |
|
|
uv run python learning_munsell/training/from_xyY/train_unified_mlp.py |
|
|
uv run python learning_munsell/training/from_xyY/train_deep_wide.py |
|
|
uv run python learning_munsell/training/from_xyY/train_mlp_attention.py |
|
|
``` |
|
|
|
|
|
**Munsell to xyY (to_xyY)** |
|
|
|
|
|
Best performing model (Multi-MLP + Multi-Error Predictor): |
|
|
|
|
|
```bash |
|
|
# Train base Multi-MLP |
|
|
uv run python learning_munsell/training/to_xyY/train_multi_mlp.py |
|
|
|
|
|
# Train multi-error predictor |
|
|
uv run python learning_munsell/training/to_xyY/train_multi_mlp_multi_error_predictor.py |
|
|
``` |
|
|
|
|
|
Other architectures: |
|
|
|
|
|
```bash |
|
|
uv run python learning_munsell/training/to_xyY/train_multi_head.py |
|
|
uv run python learning_munsell/training/to_xyY/train_multi_mlp_error_predictor.py |
|
|
uv run python learning_munsell/training/to_xyY/train_multi_head_multi_error_predictor.py |
|
|
``` |
|
|
|
|
|
Train the differentiable approximator for use in Delta-E loss: |
|
|
|
|
|
```bash |
|
|
uv run python learning_munsell/training/to_xyY/train_munsell_to_xyY_approximator.py |
|
|
``` |
|
|
|
|
|
### Hyperparameter Search |
|
|
|
|
|
```bash |
|
|
uv run python learning_munsell/training/from_xyY/hyperparameter_search_multi_head.py |
|
|
uv run python learning_munsell/training/from_xyY/hyperparameter_search_multi_head_error_predictor.py |
|
|
``` |
|
|
|
|
|
### Compare All Models |
|
|
|
|
|
```bash |
|
|
uv run python learning_munsell/comparison/from_xyY/compare_all_models.py |
|
|
uv run python learning_munsell/comparison/to_xyY/compare_all_models.py |
|
|
``` |
|
|
|
|
|
Generates comprehensive HTML reports at `reports/from_xyY/model_comparison.html` and `reports/to_xyY/model_comparison.html`. |
|
|
|
|
|
### Monitor Training |
|
|
|
|
|
**MLflow**: |
|
|
|
|
|
```bash |
|
|
uv run mlflow ui --backend-store-uri "sqlite:///mlruns.db" --port=5000 |
|
|
``` |
|
|
|
|
|
Open <http://localhost:5000> in your browser. |
|
|
|
|
|
## Directory Structure |
|
|
|
|
|
``` |
|
|
learning-munsell/ |
|
|
+-- data/ # Training data |
|
|
| +-- training_data.npz # Generated training samples |
|
|
| +-- training_data_large.npz # Large dataset (~1.4M samples) |
|
|
| +-- training_data_params.json # Generation parameters |
|
|
| +-- training_data_large_params.json |
|
|
+-- models/ # Trained models (ONNX + PyTorch) |
|
|
| +-- from_xyY/ # xyY to Munsell models (34 ONNX models) |
|
|
| | +-- multi_mlp_multi_error_predictor.onnx # BEST |
|
|
| | +-- ... (additional model variants) |
|
|
| +-- to_xyY/ # Munsell to xyY models (6 ONNX models) |
|
|
| +-- multi_mlp_multi_error_predictor.onnx # BEST |
|
|
| +-- ... (additional model variants) |
|
|
+-- learning_munsell/ # Source code |
|
|
| +-- analysis/ # Analysis scripts |
|
|
| +-- comparison/ # Model comparison scripts |
|
|
| +-- data_generation/ # Data generation scripts |
|
|
| +-- interpolation/ # Classical interpolation methods |
|
|
| +-- losses/ # Loss functions (JAX Delta-E) |
|
|
| +-- models/ # Model architecture definitions |
|
|
| +-- training/ # Model training scripts |
|
|
| +-- utilities/ # Shared utilities |
|
|
+-- docs/ # Documentation |
|
|
+-- reports/ # HTML comparison reports |
|
|
+-- logs/ # Script output logs |
|
|
+-- mlruns.db # MLflow experiment tracking database |
|
|
``` |
|
|
|
|
|
## About |
|
|
|
|
|
**Learning Munsell** by Colour Developers |
|
|
Research project for the Colour library |
|
|
<https://github.com/colour-science/colour> |
|
|
|
