| --- |
| license: mit |
| library_name: pytorch |
| pipeline_tag: image-to-image |
| tags: |
| - thermal |
| - colorization |
| - flow-matching |
| - drone |
| - dinov3 |
| --- |
| |
| # Dawn: day thermal to RGB for drone imagery |
|
|
| Checkpoints for [Dawn](https://github.com/rzx01/dawn). A daytime thermal frame (640x512, one channel) goes in, and the matching daytime RGB image comes out with brightness and color uncertainty maps. Brightness is a per-region curve of thermal intensity, color is generated by flow matching conditioned on a LoRA-tuned DINOv3 ViT-S+, and the output is assembled in CIELAB. Invented detail is accepted: thermal cannot see paint, text or lane markings. |
|
|
| ## Files |
|
|
| | File | What it is | |
| |---|---| |
| | `stage1/flow_best.pt` | Stage 1, step 18,000 (EMA), selected on CIEDE2000 over box-aligned validation pairs. Holds the LoRA adapters, velocity transformer, color head, EMA and optimizer state. | |
| | `stage1/run_config.json` | The merged config the run trained with. | |
| | `stage2/refiner_best.pt` | Optional stage 2 local refiner (2,545 parameters, refines L only). | |
|
|
| The checkpoints contain **no DINOv3 base weights**, only the LoRA adapters. The base model is the gated [facebook/dinov3-vits16plus-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vits16plus-pretrain-lvd1689m); accept its license and set `HF_TOKEN` to use these. |
|
|
| ## Results |
|
|
| Test split, 781 box-aligned pairs, NFE 2, K = 8 samples: **20.05 dB L\* PSNR and 8.31 CIEDE2000**, against a ceiling of 25.13 dB / 3.25. About 40 FPS at K = 8 on an RTX 5090. The refiner adds +0.070 dB on the fitted brightness grid (box-aligned validation); it has not been evaluated on the generated grid and is not in the headline numbers. Full numbers are in the [project README](https://github.com/rzx01/dawn#5-results). |
|
|
| ## Usage |
|
|
| ```bash |
| git clone https://github.com/rzx01/dawn && cd dawn && pip install -e . |
| hf download Arsenicc/DAWN --local-dir checkpoints |
| |
| python scripts/infer.py --checkpoint checkpoints/stage1/flow_best.pt --input frames/ --device cuda |
| python scripts/infer.py --checkpoint checkpoints/stage1/flow_best.pt \ |
| --refiner checkpoints/stage2/refiner_best.pt --input frame.jpg --device cuda |
| ``` |
|
|
| Checkpoints are tied to the training resolution (512x640 encoder input) and the token layout; loading checks both. |
|
|
| ## Training data and license |
|
|
| Trained on day pairs from [DroneVehicle](https://huggingface.co/datasets/McCheng/DroneVehicle) that survive a strict box-matched alignment filter (3,662 train pairs). No DroneVehicle images or labels are included here, and the dataset's terms are its own. The code and these checkpoints are MIT; DINOv3 has its own license. |
|
|
