--- license: bsd-3-clause language: - en tags: - python - colour - color - colour-science - color-science - colour-spaces - color-spaces - colourspace - colorspace pipeline_tag: tabular-regression library_name: onnxruntime metrics: - mae model-index: - name: from_xyY (CIE xyY to Munsell) results: - task: type: tabular-regression name: CIE xyY to Munsell Specification dataset: name: CIE xyY to Munsell Specification type: munsell-renotation metrics: - type: delta-e value: 0.52 name: Delta-E CIE2000 - type: inference_time_ms value: 0.089 name: Inference Time (ms/sample) - name: to_xyY (Munsell to CIE xyY) results: - task: type: tabular-regression name: Munsell Specification to CIE xyY dataset: name: Munsell Specification to CIE xyY type: munsell-renotation metrics: - type: delta-e value: 0.48 name: Delta-E CIE2000 - type: inference_time_ms value: 0.008 name: Inference Time (ms/sample) --- # Learning Munsell - Machine Learning for Munsell Color Conversions A project implementing machine learning-based methods for bidirectional conversion between CIE xyY colourspace values and Munsell specifications. **Two Conversion Directions:** - **from_xyY**: CIE xyY to Munsell specification - **to_xyY**: Munsell specification to CIE xyY ## Project Overview ### Objective Provide 100-1000x speedup for batch Munsell conversions compared to colour-science routines while maintaining high accuracy. ### Results **from_xyY** (CIE xyY to Munsell) — evaluated on all 2,734 REAL Munsell colors: | Model | Delta-E | Speed (ms) | |----------------------------------------------------------| ---------- | ---------- | | Colour Library (Baseline) | 0.00 | 111.90 | | **Multi-ResNet + Multi-Error Predictor (Large Dataset)** | **0.52** | 0.089 | | Multi-MLP (W+B) + Multi-Error Predictor (W+B) Large | 0.52 | 0.057 | | Multi-MLP + Multi-Error Predictor (Large Dataset) | 0.52 | 0.058 | | Multi-MLP + Multi-Error Predictor | 0.53 | 0.058 | | MLP + Error Predictor | 0.53 | 0.030 | | Multi-ResNet (Large Dataset) | 0.54 | 0.044 | | Multi-Head + Multi-Error Predictor | 0.54 | 0.042 | | Multi-Head + Multi-Error Predictor (Large Dataset) | 0.56 | 0.043 | | Deep + Wide | 0.60 | 0.074 | | Multi-Head (Large Dataset) | 0.66 | 0.013 | | Mixture of Experts | 0.80 | 0.020 | | Transformer (Large Dataset) | 0.82 | 0.123 | | Multi-MLP | 0.86 | 0.027 | | MLP + Self-Attention | 0.88 | 0.173 | | MLP (Base Only) | 1.09 | **0.007** | | Unified MLP | 1.12 | 0.072 | - **Best Accuracy**: Multi-ResNet + Multi-Error Predictor (Large Dataset) — Delta-E 0.52, 1,252x faster - **Fastest**: MLP Base Only (0.007 ms/sample) — 15,492x faster than Colour library - **Best Balance**: Multi-MLP (W+B: Weighted Boundary) + Multi-Error Predictor (W+B) Large — 1,951x faster with Delta-E 0.52 **to_xyY** (Munsell to CIE xyY) — evaluated on all 2,734 REAL Munsell colors: | Model | Delta-E | Speed (ms) | | --------------------------------------------- | ---------- | ----------- | | Colour Library (Baseline) | 0.00 | 1.27 | | **Multi-MLP (Optimized)** | **0.48** | 0.008 | | Multi-MLP (Opt) + Multi-Error Predictor (Opt) | 0.48 | 0.025 | | Multi-MLP + Multi-Error Predictor | 0.65 | 0.030 | | Multi-MLP | 0.66 | 0.016 | | Multi-MLP + Error Predictor | 0.67 | 0.018 | | Multi-Head (Optimized) | 0.71 | 0.015 | | Multi-Head | 0.78 | 0.008 | | Multi-Head + Multi-Error Predictor | 1.11 | 0.028 | | Simple MLP | 1.42 | **0.0008** | - **Best Accuracy**: Multi-MLP (Optimized) — Delta-E 0.48, 154x faster - **Fastest**: Simple MLP (0.0008 ms/sample) — 1,654x faster than Colour library ### Approach - **25+ architectures** tested for from_xyY (MLP, Multi-Head, Multi-MLP, Multi-ResNet, Transformers, Mixture of Experts) - **9 architectures** tested for to_xyY (Simple MLP, Multi-Head, Multi-MLP with error predictors) - **Two-stage models** (base + error predictor) on large dataset proved most effective - **Best model**: Multi-ResNet + Multi-Error Predictor (Large Dataset) with Delta-E 0.52 - **Training data**: ~1.4M samples from dense xyY grid with boundary refinement and forward Munsell sampling - **Deployment**: ONNX format with ONNX Runtime For detailed architecture comparisons, model benchmarks, training pipeline details, and experimental results, see [docs/learning_munsell.md](docs/learning_munsell.md). ## Installation **Dependencies (Runtime)**: - numpy >= 2.0 - onnxruntime >= 1.16 **Dependencies (Training)**: - torch >= 2.0 - scikit-learn >= 1.3 - matplotlib >= 3.9 - mlflow >= 2.10 - optuna >= 3.0 - colour-science >= 0.4.7 - click >= 8.0 - onnx >= 1.15 - onnxscript >= 0.5.6 - tqdm >= 4.66 - jax >= 0.4.20 - jaxlib >= 0.4.20 - flax >= 0.10.7 - optax >= 0.2.6 - scipy >= 1.12 - tensorboard >= 2.20 From the project root: ```bash cd learning-munsell # Install all dependencies (creates virtual environment automatically) uv sync ``` ## Usage ### Generate Training Data ```bash uv run python learning_munsell/data_generation/generate_training_data.py ``` **Note**: This step is computationally expensive (uses iterative algorithm for ground truth). ### Train Models **xyY to Munsell (from_xyY)** Best performing model (Multi-ResNet + Multi-Error Predictor on Large Dataset): ```bash # Train base Multi-ResNet on large dataset (~1.4M samples) uv run python learning_munsell/training/from_xyY/train_multi_resnet_large.py # Train multi-error predictor uv run python learning_munsell/training/from_xyY/train_multi_resnet_error_predictor_large.py ``` Alternative (Multi-Head architecture): ```bash uv run python learning_munsell/training/from_xyY/train_multi_head_large.py uv run python learning_munsell/training/from_xyY/train_multi_head_multi_error_predictor_large.py ``` Other architectures: ```bash uv run python learning_munsell/training/from_xyY/train_unified_mlp.py uv run python learning_munsell/training/from_xyY/train_multi_mlp.py uv run python learning_munsell/training/from_xyY/train_mlp_attention.py uv run python learning_munsell/training/from_xyY/train_deep_wide.py uv run python learning_munsell/training/from_xyY/train_ft_transformer.py ``` **Munsell to xyY (to_xyY)** Best performing model (Multi-MLP Optimized): ```bash uv run python learning_munsell/training/to_xyY/train_multi_mlp.py uv run python learning_munsell/training/to_xyY/train_multi_head.py uv run python learning_munsell/training/to_xyY/train_multi_mlp_multi_error_predictor.py uv run python learning_munsell/training/to_xyY/train_multi_mlp_error_predictor.py uv run python learning_munsell/training/to_xyY/train_multi_head_multi_error_predictor.py ``` Train the differentiable approximator for use in Delta-E loss: ```bash uv run python learning_munsell/training/to_xyY/train_munsell_to_xyY_approximator.py ``` ### Hyperparameter Search ```bash uv run python learning_munsell/training/from_xyY/hyperparameter_search_multi_head.py uv run python learning_munsell/training/from_xyY/hyperparameter_search_multi_head_error_predictor.py ``` ### Compare All Models ```bash uv run python learning_munsell/comparison/from_xyY/compare_all_models.py ``` Generates comprehensive HTML report at `reports/from_xyY/model_comparison.html`. ### Monitor Training **MLflow**: ```bash uv run mlflow ui --backend-store-uri "sqlite:///mlruns.db" --port=5000 ``` Open in your browser. ## Directory Structure ``` learning-munsell/ +-- data/ # Training data | +-- training_data.npz # Generated training samples | +-- training_data_large.npz # Large dataset (~1.4M samples) | +-- training_data_params.json # Generation parameters | +-- training_data_large_params.json +-- models/ # Trained models (ONNX + PyTorch) | +-- from_xyY/ # xyY to Munsell models (25+ ONNX models) | | +-- multi_resnet_error_predictor_large.onnx # BEST | | +-- ... (additional model variants) | +-- to_xyY/ # Munsell to xyY models (9 ONNX models) | +-- multi_mlp_optimized.onnx # BEST | +-- ... (additional model variants) +-- learning_munsell/ # Source code | +-- analysis/ # Analysis scripts | +-- comparison/ # Model comparison scripts | +-- data_generation/ # Data generation scripts | +-- interpolation/ # Classical interpolation methods | +-- losses/ # Loss functions (JAX Delta-E) | +-- models/ # Model architecture definitions | +-- training/ # Model training scripts | +-- utilities/ # Shared utilities +-- docs/ # Documentation +-- reports/ # HTML comparison reports +-- logs/ # Script output logs +-- mlruns.db # MLflow experiment tracking database ``` ## About **Learning Munsell** by Colour Developers Research project for the Colour library