colour-science/learning-munsell

Learning Munsell - Machine Learning for Munsell Color Conversions

A project implementing machine learning-based methods for bidirectional conversion between CIE xyY colourspace values and Munsell specifications.

Two Conversion Directions:

• from_xyY: CIE xyY to Munsell specification
• to_xyY: Munsell specification to CIE xyY

Project Overview

Objective

Provide 100-1000x speedup for batch Munsell conversions compared to colour-science routines while maintaining high accuracy.

Results

from_xyY (CIE xyY to Munsell) — evaluated on all 2,734 REAL Munsell colors:

Model	Delta-E	Speed (ms)
Colour Library (Baseline)	0.00	111.90
Multi-ResNet + Multi-Error Predictor (Large Dataset)	0.52	0.089
Multi-MLP (W+B) + Multi-Error Predictor (W+B) Large	0.52	0.057
Multi-MLP + Multi-Error Predictor (Large Dataset)	0.52	0.058
Multi-MLP + Multi-Error Predictor	0.53	0.058
MLP + Error Predictor	0.53	0.030
Multi-ResNet (Large Dataset)	0.54	0.044
Multi-Head + Multi-Error Predictor	0.54	0.042
Multi-Head + Multi-Error Predictor (Large Dataset)	0.56	0.043
Deep + Wide	0.60	0.074
Multi-Head (Large Dataset)	0.66	0.013
Mixture of Experts	0.80	0.020
Transformer (Large Dataset)	0.82	0.123
Multi-MLP	0.86	0.027
MLP + Self-Attention	0.88	0.173
MLP (Base Only)	1.09	0.007
Unified MLP	1.12	0.072

• Best Accuracy: Multi-ResNet + Multi-Error Predictor (Large Dataset) — Delta-E 0.52, 1,252x faster
• Fastest: MLP Base Only (0.007 ms/sample) — 15,492x faster than Colour library
• Best Balance: Multi-MLP (W+B: Weighted Boundary) + Multi-Error Predictor (W+B) Large — 1,951x faster with Delta-E 0.52

to_xyY (Munsell to CIE xyY) — evaluated on all 2,734 REAL Munsell colors:

Model	Delta-E	Speed (ms)
Colour Library (Baseline)	0.00	1.27
Multi-MLP (Optimized)	0.48	0.008
Multi-MLP (Opt) + Multi-Error Predictor (Opt)	0.48	0.025
Multi-MLP + Multi-Error Predictor	0.65	0.030
Multi-MLP	0.66	0.016
Multi-MLP + Error Predictor	0.67	0.018
Multi-Head (Optimized)	0.71	0.015
Multi-Head	0.78	0.008
Multi-Head + Multi-Error Predictor	1.11	0.028
Simple MLP	1.42	0.0008

• Best Accuracy: Multi-MLP (Optimized) — Delta-E 0.48, 154x faster
• Fastest: Simple MLP (0.0008 ms/sample) — 1,654x faster than Colour library

Approach

• 25+ architectures tested for from_xyY (MLP, Multi-Head, Multi-MLP, Multi-ResNet, Transformers, Mixture of Experts)
• 9 architectures tested for to_xyY (Simple MLP, Multi-Head, Multi-MLP with error predictors)
• Two-stage models (base + error predictor) on large dataset proved most effective
• Best model: Multi-ResNet + Multi-Error Predictor (Large Dataset) with Delta-E 0.52
• Training data: ~1.4M samples from dense xyY grid with boundary refinement and forward Munsell sampling
• Deployment: ONNX format with ONNX Runtime

For detailed architecture comparisons, model benchmarks, training pipeline details, and experimental results, see docs/learning_munsell.md.

Installation

Dependencies (Runtime):

• numpy >= 2.0
• onnxruntime >= 1.16

Dependencies (Training):

• torch >= 2.0
• scikit-learn >= 1.3
• matplotlib >= 3.9
• mlflow >= 2.10
• optuna >= 3.0
• colour-science >= 0.4.7
• click >= 8.0
• onnx >= 1.15
• onnxscript >= 0.5.6
• tqdm >= 4.66
• jax >= 0.4.20
• jaxlib >= 0.4.20
• flax >= 0.10.7
• optax >= 0.2.6
• scipy >= 1.12
• tensorboard >= 2.20

From the project root:

cd learning-munsell

# Install all dependencies (creates virtual environment automatically)
uv sync

Usage

Generate Training Data

uv run python learning_munsell/data_generation/generate_training_data.py

Note: This step is computationally expensive (uses iterative algorithm for ground truth).

Train Models

xyY to Munsell (from_xyY)

Best performing model (Multi-ResNet + Multi-Error Predictor on Large Dataset):

# Train base Multi-ResNet on large dataset (~1.4M samples)
uv run python learning_munsell/training/from_xyY/train_multi_resnet_large.py

# Train multi-error predictor
uv run python learning_munsell/training/from_xyY/train_multi_resnet_error_predictor_large.py

Alternative (Multi-Head architecture):

uv run python learning_munsell/training/from_xyY/train_multi_head_large.py
uv run python learning_munsell/training/from_xyY/train_multi_head_multi_error_predictor_large.py

Other architectures:

uv run python learning_munsell/training/from_xyY/train_unified_mlp.py
uv run python learning_munsell/training/from_xyY/train_multi_mlp.py
uv run python learning_munsell/training/from_xyY/train_mlp_attention.py
uv run python learning_munsell/training/from_xyY/train_deep_wide.py
uv run python learning_munsell/training/from_xyY/train_ft_transformer.py

Munsell to xyY (to_xyY)

Best performing model (Multi-MLP Optimized):

uv run python learning_munsell/training/to_xyY/train_multi_mlp.py
uv run python learning_munsell/training/to_xyY/train_multi_head.py
uv run python learning_munsell/training/to_xyY/train_multi_mlp_multi_error_predictor.py
uv run python learning_munsell/training/to_xyY/train_multi_mlp_error_predictor.py
uv run python learning_munsell/training/to_xyY/train_multi_head_multi_error_predictor.py

Train the differentiable approximator for use in Delta-E loss:

uv run python learning_munsell/training/to_xyY/train_munsell_to_xyY_approximator.py

Hyperparameter Search

uv run python learning_munsell/training/from_xyY/hyperparameter_search_multi_head.py
uv run python learning_munsell/training/from_xyY/hyperparameter_search_multi_head_error_predictor.py

Compare All Models

uv run python learning_munsell/comparison/from_xyY/compare_all_models.py

Generates comprehensive HTML report at reports/from_xyY/model_comparison.html.

Monitor Training

MLflow:

uv run mlflow ui --backend-store-uri "sqlite:///mlruns.db" --port=5000

Open http://localhost:5000 in your browser.

Directory Structure

learning-munsell/
+-- data/                          # Training data
|   +-- training_data.npz          # Generated training samples
|   +-- training_data_large.npz    # Large dataset (~1.4M samples)
|   +-- training_data_params.json  # Generation parameters
|   +-- training_data_large_params.json
+-- models/                        # Trained models (ONNX + PyTorch)
|   +-- from_xyY/                  # xyY to Munsell models (25+ ONNX models)
|   |   +-- multi_resnet_error_predictor_large.onnx  # BEST
|   |   +-- ... (additional model variants)
|   +-- to_xyY/                    # Munsell to xyY models (9 ONNX models)
|       +-- multi_mlp_optimized.onnx  # BEST
|       +-- ... (additional model variants)
+-- learning_munsell/              # Source code
|   +-- analysis/                  # Analysis scripts
|   +-- comparison/                # Model comparison scripts
|   +-- data_generation/           # Data generation scripts
|   +-- interpolation/             # Classical interpolation methods
|   +-- losses/                    # Loss functions (JAX Delta-E)
|   +-- models/                    # Model architecture definitions
|   +-- training/                  # Model training scripts
|   +-- utilities/                 # Shared utilities
+-- docs/                          # Documentation
+-- reports/                       # HTML comparison reports
+-- logs/                          # Script output logs
+-- mlruns.db                      # MLflow experiment tracking database

About

Learning Munsell by Colour Developers Research project for the Colour library https://github.com/colour-science/colour