outputs/eval/metrics_pilot.log

Setting up [LPIPS] perceptual loss: trunk [alex], v[0.1], spatial [off]
/usr/local/lib/python3.11/dist-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/usr/local/lib/python3.11/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=AlexNet_Weights.IMAGENET1K_V1`. You can also use `weights=AlexNet_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Loading model from: /usr/local/lib/python3.11/dist-packages/lpips/weights/v0.1/alex.pth
/usr/local/lib/python3.11/dist-packages/open_clip/factory.py:450: UserWarning: QuickGELU mismatch between final model config (quick_gelu=False) and pretrained tag 'openai' (quick_gelu=True).
  warnings.warn(
W0613 19:28:57.813000 28952 torch/utils/_pytree.py:630] <enum 'KernelPreference'> is an Enum subclass and is now natively supported by torch.compile as an opaque value type. Calling register_constant() on Enum subclasses is deprecated and will be an error in a future release.
W0613 19:28:57.836000 28952 torch/utils/_pytree.py:630] <enum 'ScaleCalculationMode'> is an Enum subclass and is now natively supported by torch.compile as an opaque value type. Calling register_constant() on Enum subclasses is deprecated and will be an error in a future release.
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
WARNING:huggingface_hub.utils._http:Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.

Loading weights:   0%|          | 0/910 [00:00<?, ?it/s]
Loading weights:  82%|████████▏ | 747/910 [00:00<00:00, 7468.23it/s]
Loading weights: 100%|██████████| 910/910 [00:00<00:00, 7541.25it/s]
--- teacher (64 imgs) ---
  {'PickScore': 21.7849, 'CLIP': 31.1447, 'LPIPS': None, 'PSNR': None, 'FID_vs_real': np.float64(219.406), 'FID_vs_teacher': None}
--- pilot_ours128_real (64 imgs) ---
  {'PickScore': 21.8082, 'CLIP': 31.1317, 'LPIPS': 0.1688, 'PSNR': 18.9134, 'FID_vs_real': np.float64(217.655), 'FID_vs_teacher': np.float64(52.606)}
--- pilot_fq128 (64 imgs) ---
  {'PickScore': 21.7575, 'CLIP': 31.3089, 'LPIPS': 0.1702, 'PSNR': 18.9429, 'FID_vs_real': np.float64(217.17), 'FID_vs_teacher': np.float64(54.883)}
--- pilot_fq0 (64 imgs) ---
  {'PickScore': 21.7593, 'CLIP': 31.1597, 'LPIPS': 0.2095, 'PSNR': 17.835, 'FID_vs_real': np.float64(216.649), 'FID_vs_teacher': np.float64(65.268)}

=== SUMMARY (outputs/eval/metrics.json) ===
model                          PickScore↑ CLIP↑   LPIPS↓  PSNR↑   FIDreal↓  FIDteach↓
teacher                        21.7849    31.1447 None    None    219.406   None     
pilot_ours128_real             21.8082    31.1317 0.1688  18.9134 217.655   52.606   
pilot_fq128                    21.7575    31.3089 0.1702  18.9429 217.17    54.883   
pilot_fq0                      21.7593    31.1597 0.2095  17.835  216.649   65.268