Add model card, checksums and brightness head
Browse files- README.md +86 -0
- brightness/model.pt +3 -0
- checksums.json +14 -0
README.md
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pipeline_tag: image-to-image
|
| 3 |
+
tags:
|
| 4 |
+
- pytorch
|
| 5 |
+
- hirise
|
| 6 |
+
- mars
|
| 7 |
+
- remote-sensing
|
| 8 |
+
- flow-matching
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# HiPredict Diffusion
|
| 12 |
+
|
| 13 |
+
Bidirectional conditional flow-matching checkpoints for reconstructing HiRISE RED4 from IR10, or IR10 from RED4. Use these checkpoints with the [HiPredict Python package](https://github.com/EricKing22/HiPredict).
|
| 14 |
+
|
| 15 |
+
## Checkpoints
|
| 16 |
+
|
| 17 |
+
| File | Model | Training steps | Gradient-loss weight |
|
| 18 |
+
| --- | --- | --- | --- |
|
| 19 |
+
| `FM_Condition/latest.pt` | BidirectionalFM, Condition mode | 100,000 | 0.0 |
|
| 20 |
+
| `FM_Condition_gradient/latest.pt` | BidirectionalFM, Condition mode | 100,000 | 0.1 |
|
| 21 |
+
| `brightness/model.pt` | Auxiliary RED3 boundary-residual brightness head | — | — |
|
| 22 |
+
|
| 23 |
+
Choose either FM checkpoint as the main model. Both support `ir2red` and `red2ir`. The brightness head is optional: the default boundary-based correction does not need it.
|
| 24 |
+
|
| 25 |
+
Both main checkpoints include model weights, EMA weights, optimizer state, learning-rate scheduler state, and model/training settings. Each file is approximately 2.05 GB. The brightness head is approximately 32 KB.
|
| 26 |
+
|
| 27 |
+
## Installation
|
| 28 |
+
|
| 29 |
+
Use a HiPredict checkout containing `hipredict.diffusion` and install its diffusion dependencies from the checkout root:
|
| 30 |
+
|
| 31 |
+
```bash
|
| 32 |
+
pip install ".[diffusion]"
|
| 33 |
+
```
|
| 34 |
+
|
| 35 |
+
For evaluation with HiPredict's shared metrics, install `.[diffusion,postprocess]` instead.
|
| 36 |
+
|
| 37 |
+
## Download and load
|
| 38 |
+
|
| 39 |
+
```python
|
| 40 |
+
from huggingface_hub import hf_hub_download
|
| 41 |
+
from hipredict.diffusion import load_checkpoint
|
| 42 |
+
|
| 43 |
+
checkpoint = hf_hub_download(
|
| 44 |
+
repo_id="syding9/HiPredict_Diffusion",
|
| 45 |
+
filename="FM_Condition/latest.pt",
|
| 46 |
+
)
|
| 47 |
+
model, model_config, training_config, step = load_checkpoint(
|
| 48 |
+
checkpoint, device="cuda"
|
| 49 |
+
)
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
Select `FM_Condition_gradient/latest.pt` for the gradient-loss model.
|
| 53 |
+
|
| 54 |
+
## HiPredict API
|
| 55 |
+
|
| 56 |
+
The HiPredict API adapter reconstructs RED4 (`ir2red`), on patches or full strips. Large images are predicted with overlapping tiles.
|
| 57 |
+
|
| 58 |
+
```python
|
| 59 |
+
from hipredict.diffusion import build_hipredict_predictor
|
| 60 |
+
|
| 61 |
+
predictor, config, step = build_hipredict_predictor(checkpoint, device="cuda")
|
| 62 |
+
samples = predictor.load("files/data_record_evals.csv", mode="patches")
|
| 63 |
+
predictions = predictor.predict(samples)
|
| 64 |
+
```
|
| 65 |
+
|
| 66 |
+
For `red2ir`, use the diffusion inference command with the downloaded checkpoint path:
|
| 67 |
+
|
| 68 |
+
```bash
|
| 69 |
+
python -m hipredict.diffusion.inference.inference --ckpt_path /path/to/latest.pt --input_path observation_RED4.npy --red3_path observation_RED3.npy --direction red2ir --output_dir outputs/inference/
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
+
## Brightness correction
|
| 73 |
+
|
| 74 |
+
The default correction uses the RED3 boundary. To use the learned residual brightness head for dataset inference or evaluation, download `brightness/model.pt`, select `--dc_correct head`, and pass its local path as `--head_ckpt`.
|
| 75 |
+
|
| 76 |
+
## Model configuration
|
| 77 |
+
|
| 78 |
+
Both FM checkpoints use a 128-channel base width, three residual blocks per level, Sobel edge conditioning, and noise-conditioned flow matching. Their normalization settings are stored in the checkpoint:
|
| 79 |
+
|
| 80 |
+
- Center: `0.103687465`
|
| 81 |
+
- Scale: `0.04842728`
|
| 82 |
+
- Normalized training values clipped to `[-5, 5]`
|
| 83 |
+
|
| 84 |
+
HiPredict reads these settings when loading the checkpoint. Evaluation uses original, unclipped physical targets. The checkpoint format is `hipredict.diffusion`, version `1`.
|
| 85 |
+
|
| 86 |
+
File SHA-256 digests are recorded in `checksums.json`.
|
brightness/model.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4bc20b87fe967cbb9856395594c4db5407c474c7bddf2a2e2158d38f7056c131
|
| 3 |
+
size 32509
|
checksums.json
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"FM_Condition/latest.pt": {
|
| 3 |
+
"sha256": "8aad7235ef7e117bf1bbf3ef55b3e0a76f35b85d884f2f00b1588c0075c26f4d",
|
| 4 |
+
"size_bytes": 2048582622
|
| 5 |
+
},
|
| 6 |
+
"FM_Condition_gradient/latest.pt": {
|
| 7 |
+
"sha256": "2c1698b8cfb9ad0bc058ec0928a7d4770acefcd10d51da95c39315b44f056484",
|
| 8 |
+
"size_bytes": 2048582622
|
| 9 |
+
},
|
| 10 |
+
"brightness/model.pt": {
|
| 11 |
+
"sha256": "4bc20b87fe967cbb9856395594c4db5407c474c7bddf2a2e2158d38f7056c131",
|
| 12 |
+
"size_bytes": 32509
|
| 13 |
+
}
|
| 14 |
+
}
|