supli6669 commited on
Commit ·
d6d0cbd
1
Parent(s): c0b25cc
feat: add 20 specialized project skills for image quality enhancement and CPU performance
Browse files- .agents/skills/arcface-identity-loss-tuning/SKILL.md +30 -0
- .agents/skills/blind-degradation-pipeline/SKILL.md +16 -0
- .agents/skills/chromatic-aberration-correction/SKILL.md +30 -0
- .agents/skills/eye-and-teeth-enhancement/SKILL.md +42 -0
- .agents/skills/face-landmark-alignment-5pt/SKILL.md +27 -0
- .agents/skills/face-parsing-segmentation-masks/SKILL.md +32 -0
- .agents/skills/fast-cpu-vectorization-numpy/SKILL.md +14 -0
- .agents/skills/frequency-separation-skin-grain/SKILL.md +38 -0
- .agents/skills/guided-filter-detail-enhancement/SKILL.md +22 -0
- .agents/skills/hdr-tone-mapping-exposure/SKILL.md +29 -0
- .agents/skills/huggingface-space-docker-deploy/SKILL.md +25 -0
- .agents/skills/image-quality-metrics-eval/SKILL.md +16 -0
- .agents/skills/lab-clahe-color-grading/SKILL.md +29 -0
- .agents/skills/lmdb-dataset-binary-storage/SKILL.md +25 -0
- .agents/skills/noise-estimation-denoising/SKILL.md +15 -0
- .agents/skills/onnx-static-int8-quantization/SKILL.md +34 -0
- .agents/skills/streamlit-side-by-side-visualizer/SKILL.md +14 -0
- .agents/skills/streamlit-thread-state-guidelines/SKILL.md +1 -1
- .agents/skills/super-resolution-tile-stitching/SKILL.md +29 -0
- .agents/skills/unsharp-masking-deconvolution/SKILL.md +25 -0
- .agents/skills/vqgan-codebook-fine-tuning/SKILL.md +14 -0
- models/CodeFormer/options/cf_runtime_szvghq3h.yml +166 -0
- models/CodeFormer/options/cf_runtime_vmvjxo41.yml +166 -0
- wink_enhancer.py +22 -13
.agents/skills/arcface-identity-loss-tuning/SKILL.md
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: arcface-identity-loss-tuning
|
| 3 |
+
description: Integrating ArcFace/CosFace identity feature extractors into PyTorch GAN loss pipelines to preserve facial identity during fine-tuning.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# ArcFace Identity Loss Integration & Fine-Tuning Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Standard pixel (L1) and perceptual (LPIPS) losses focus on surface appearance, which can drift human identity features during high-fidelity face restoration. ArcFace extracts deep 512-dimensional face embedding vectors. Computing cosine distance between restored and target embeddings guarantees identity preservation.
|
| 10 |
+
|
| 11 |
+
## Mathematical Formulation
|
| 12 |
+
$$\mathcal{L}_{id} = 1 - rac{ ext{ArcFace}(I_{rec}) \cdot ext{ArcFace}(I_{gt})}{\| ext{ArcFace}(I_{rec})\| \| ext{ArcFace}(I_{gt})\|}$$
|
| 13 |
+
|
| 14 |
+
## Code Snippet
|
| 15 |
+
```python
|
| 16 |
+
import torch
|
| 17 |
+
import torch.nn.functional as F
|
| 18 |
+
|
| 19 |
+
class ArcFaceLoss(torch.nn.Module):
|
| 20 |
+
def __init__(self, net):
|
| 21 |
+
super().__init__()
|
| 22 |
+
self.net = net.eval()
|
| 23 |
+
for p in self.net.parameters():
|
| 24 |
+
p.requires_grad = False
|
| 25 |
+
|
| 26 |
+
def forward(self, pred, target):
|
| 27 |
+
emb_pred = F.normalize(self.net(pred), p=2, dim=1)
|
| 28 |
+
emb_gt = F.normalize(self.net(target), p=2, dim=1)
|
| 29 |
+
return 1.0 - torch.sum(emb_pred * emb_gt, dim=1).mean()
|
| 30 |
+
```
|
.agents/skills/blind-degradation-pipeline/SKILL.md
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: blind-degradation-pipeline
|
| 3 |
+
description: Synthesizing real-world image degradation pipelines including anisotropic Gaussian blur, motion kernels, Poisson/Gaussian noise, JPEG compression artifacts, and multi-stage downsampling.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Blind Degradation Pipeline Modeling Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Training robust face restoration models requires simulating realistic real-world corruptions. A multi-stage degradation pipeline combines blur, noise, downsampling, and JPEG compression in randomized orders.
|
| 10 |
+
|
| 11 |
+
## Degradation Order
|
| 12 |
+
1. **Blur**: Anisotropic Gaussian Blur ($\sigma \in [0.2, 3.0]$) + Motion blur kernel.
|
| 13 |
+
2. **Downsampling**: Random resize (area, bilinear, bicubic) down to scale factor $r \in [1, 8]$.
|
| 14 |
+
3. **Noise**: Additive Gaussian noise ($\sigma \in [0, 30]$) + Poisson noise.
|
| 15 |
+
4. **JPEG Compression**: Quality factor $Q \in [10, 70]$.
|
| 16 |
+
5. **Final Downscale**: Resize to network input shape ($512 imes 512$).
|
.agents/skills/chromatic-aberration-correction/SKILL.md
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: chromatic-aberration-correction
|
| 3 |
+
description: Detection and correction of lateral and longitudinal chromatic aberration (color fringing), radial channel alignment, and lens distortion repair.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Chromatic Aberration & Color Fringing Correction Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Chromatic aberration produces red/cyan or blue/yellow color fringes along high-contrast edges. Radial channel shift aligns Red and Blue channels to Green.
|
| 10 |
+
|
| 11 |
+
## Code Example
|
| 12 |
+
```python
|
| 13 |
+
import cv2
|
| 14 |
+
import numpy as np
|
| 15 |
+
|
| 16 |
+
def correct_chromatic_aberration(img_bgr):
|
| 17 |
+
b, g, r = cv2.split(img_bgr)
|
| 18 |
+
# Estimate warp matrix between channels using ECC or phase correlation
|
| 19 |
+
warp_mode = cv2.MOTION_TRANSLATION
|
| 20 |
+
warp_matrix = np.eye(2, 3, dtype=np.float32)
|
| 21 |
+
criteria = (cv2.TERMCRITERIA_EPS | cv2.TERMCRITERIA_COUNT, 50, 1e-4)
|
| 22 |
+
|
| 23 |
+
try:
|
| 24 |
+
_, warp_matrix_r = cv2.findTransformECC(g, r, warp_matrix.copy(), warp_mode, criteria)
|
| 25 |
+
r_corr = cv2.warpAffine(r, warp_matrix_r, (g.shape[1], g.shape[0]), flags=cv2.INTER_LINEAR + cv2.WARP_INVERSE_MAP)
|
| 26 |
+
except cv2.error:
|
| 27 |
+
r_corr = r
|
| 28 |
+
|
| 29 |
+
return cv2.merge((b, g, r_corr))
|
| 30 |
+
```
|
.agents/skills/eye-and-teeth-enhancement/SKILL.md
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: eye-and-teeth-enhancement
|
| 3 |
+
description: Facial parsing targeted enhancements for boosting iris clarity, catchlight sparkle, ocular contrast, and natural teeth brightening without over-bleaching.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Eye & Teeth Targeted Enhancement Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Eyes and teeth are central focal points in portrait photography. This skill targets parsed eye and mouth masks to boost iris sharpness, add catchlight clarity, and gently brighten teeth while maintaining natural skin balance.
|
| 10 |
+
|
| 11 |
+
## Procedures
|
| 12 |
+
1. **Iris & Ocular Boost**:
|
| 13 |
+
- Extract eye mask from face parsing output.
|
| 14 |
+
- Apply localized CLAHE and high-pass sharpening on the ocular region ($1.2 imes$ scale).
|
| 15 |
+
2. **Teeth Brightening**:
|
| 16 |
+
- Isolate inner mouth/teeth mask.
|
| 17 |
+
- Convert to HSV space; slightly desaturate yellow ($S imes 0.85$) and boost luminance ($V imes 1.08$).
|
| 18 |
+
|
| 19 |
+
## Code Example
|
| 20 |
+
```python
|
| 21 |
+
import cv2
|
| 22 |
+
import numpy as np
|
| 23 |
+
|
| 24 |
+
def enhance_eyes_and_teeth(img_bgr, eye_mask, teeth_mask):
|
| 25 |
+
out = img_bgr.copy()
|
| 26 |
+
|
| 27 |
+
# Eye sharpness boost
|
| 28 |
+
if np.any(eye_mask > 0):
|
| 29 |
+
eyes_sharp = cv2.addWeighted(img_bgr, 1.3, cv2.GaussianBlur(img_bgr, (0,0), 2.0), -0.3, 0)
|
| 30 |
+
mask_eye_f = (eye_mask.astype(np.float32) / 255.0)[:, :, None]
|
| 31 |
+
out = np.uint8(out * (1 - mask_eye_f) + eyes_sharp * mask_eye_f)
|
| 32 |
+
|
| 33 |
+
# Teeth whitening in HSV
|
| 34 |
+
if np.any(teeth_mask > 0):
|
| 35 |
+
hsv = cv2.cvtColor(out, cv2.COLOR_BGR2HSV).astype(np.float32)
|
| 36 |
+
hsv[:, :, 1] *= np.where(teeth_mask > 0, 0.85, 1.0) # desaturate yellow
|
| 37 |
+
hsv[:, :, 2] *= np.where(teeth_mask > 0, 1.08, 1.0) # boost brightness
|
| 38 |
+
hsv = np.clip(hsv, 0, 255).astype(np.uint8)
|
| 39 |
+
out = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)
|
| 40 |
+
|
| 41 |
+
return out
|
| 42 |
+
```
|
.agents/skills/face-landmark-alignment-5pt/SKILL.md
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: face-landmark-alignment-5pt
|
| 3 |
+
description: Extracting 5-point facial landmarks (eyes, nose, mouth corners), computing similarity transformation matrices, $512 imes 512$ alignment, and reverse warp-back.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# 5-Point Facial Landmark Alignment & Inverse Warp Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Face restoration models require normalized face crops aligned to standard 5-point landmark canonical coordinates (left eye, right eye, nose tip, left mouth corner, right mouth corner). After restoration, the inverse affine transformation warps the crop back to the exact spatial location on the original frame.
|
| 10 |
+
|
| 11 |
+
## Canonical 512x512 Landmark Anchors
|
| 12 |
+
- Left Eye: $(192.98, 239.95)$
|
| 13 |
+
- Right Eye: $(318.55, 240.14)$
|
| 14 |
+
- Nose Tip: $(256.63, 314.02)$
|
| 15 |
+
- Left Mouth Corner: $(201.26, 371.41)$
|
| 16 |
+
- Right Mouth Corner: $(313.00, 371.19)$
|
| 17 |
+
|
| 18 |
+
## Code Snippet
|
| 19 |
+
```python
|
| 20 |
+
import cv2
|
| 21 |
+
import numpy as np
|
| 22 |
+
|
| 23 |
+
def get_similarity_transform(src_pts, dst_pts):
|
| 24 |
+
# Computes 2x3 affine matrix for similarity transform (scale, rotation, translation)
|
| 25 |
+
tfm = cv2.estimateAffinePartial2D(src_pts, dst_pts)[0]
|
| 26 |
+
return tfm
|
| 27 |
+
```
|
.agents/skills/face-parsing-segmentation-masks/SKILL.md
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: face-parsing-segmentation-masks
|
| 3 |
+
description: Facial component segmentation using ParseNet/BiSeNet, extracting eye, lip, skin, and hair masks for targeted post-processing and seamless edge blending.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Face Parsing & Segmentation Masks Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Face parsing segments a face image into semantic components: eyes, eyebrows, nose, mouth/lips, skin, hair, and background. Using these parsing masks allows targeted visual enhancements (e.g. skin grain injection, eye sharpening, lip tone balancing) without affecting non-target regions.
|
| 10 |
+
|
| 11 |
+
## Key Concepts
|
| 12 |
+
1. **Model Architecture**: BiSeNet or ParseNet trained on CelebAMask-HQ. Outputs 19 semantic channels.
|
| 13 |
+
2. **Mask Generation**:
|
| 14 |
+
- `skin_mask = (parsing == 1)`
|
| 15 |
+
- `eye_mask = (parsing == 4) | (parsing == 5)`
|
| 16 |
+
- `mouth_mask = (parsing == 11) | (parsing == 12) | (parsing == 13)`
|
| 17 |
+
3. **Blending & Dilated Boundaries**:
|
| 18 |
+
- Apply Gaussian blur (`cv2.GaussianBlur`) to mask edges for soft alpha transitions.
|
| 19 |
+
- Use morphological erosion (`cv2.erode`) to avoid seam artifacts near hairline or jawline.
|
| 20 |
+
|
| 21 |
+
## Code Example
|
| 22 |
+
```python
|
| 23 |
+
import cv2
|
| 24 |
+
import numpy as np
|
| 25 |
+
|
| 26 |
+
def extract_parsing_masks(parsing_out):
|
| 27 |
+
# parsing_out shape: (512, 512) containing class indices 0..18
|
| 28 |
+
skin = np.uint8((parsing_out == 1) * 255)
|
| 29 |
+
eyes = np.uint8(((parsing_out == 4) | (parsing_out == 5)) * 255)
|
| 30 |
+
lips = np.uint8(((parsing_out == 11) | (parsing_out == 12) | (parsing_out == 13)) * 255)
|
| 31 |
+
return skin, eyes, lips
|
| 32 |
+
```
|
.agents/skills/fast-cpu-vectorization-numpy/SKILL.md
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: fast-cpu-vectorization-numpy
|
| 3 |
+
description: Optimizing Python image pipelines with C-vectorized NumPy operations, OpenCV inplace buffer operations, and OpenMP multi-threading controls for <20ms CPU latency.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Fast CPU Vectorization & NumPy Optimization Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Running image post-processing in Python loops creates severe latency bottlenecks. Vectorizing image math with NumPy broadcasting, OpenCV inplace buffers (`dst=out`), and contiguous memory layouts ensures lightning-fast execution (<20ms per face).
|
| 10 |
+
|
| 11 |
+
## Golden Rules
|
| 12 |
+
1. **Never Python Loops over Pixels**: Always use vectorized slice operations (`img[:, :, 0]`).
|
| 13 |
+
2. **Inplace OpenCV Buffers**: Pass output buffer directly `cv2.addWeighted(img1, 0.5, img2, 0.5, 0, dst=buffer)`.
|
| 14 |
+
3. **Memory Contiguity**: Call `np.ascontiguousarray()` before passing arrays to C/C++ extensions.
|
.agents/skills/frequency-separation-skin-grain/SKILL.md
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: frequency-separation-skin-grain
|
| 3 |
+
description: High/low frequency image decomposition, extracting authentic skin pore textures from original images, and blending texture back onto neural network face crops to eliminate plastic skin.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Frequency Separation & Skin Grain Retention Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Neural face restoration models (like CodeFormer) can produce over-smoothed, 'plastic' or 'soapy' skin textures. Frequency separation splits an image into Low Frequency (tones, color, shading) and High Frequency (fine pores, micro-textures, wrinkles). By injecting original high-frequency details onto the restored face crop, skin looks natural and hyper-realistic.
|
| 10 |
+
|
| 11 |
+
## Key Concepts
|
| 12 |
+
1. **Low Frequency Extraction**: Apply Gaussian blur (`sigma = 2.0 to 4.0`).
|
| 13 |
+
2. **High Frequency Extraction**: Subtract Low Frequency from original image: $HF = Original - LF + 128$.
|
| 14 |
+
3. **Texture Injection**: Blend original high frequency onto the restored face using parsing skin masks.
|
| 15 |
+
|
| 16 |
+
## Vectorized NumPy Implementation
|
| 17 |
+
```python
|
| 18 |
+
import cv2
|
| 19 |
+
import numpy as np
|
| 20 |
+
|
| 21 |
+
def inject_skin_grain(orig_crop, restored_crop, skin_mask, weight=0.15):
|
| 22 |
+
# Convert to float32
|
| 23 |
+
orig_f = orig_crop.astype(np.float32)
|
| 24 |
+
rest_f = restored_crop.astype(np.float32)
|
| 25 |
+
|
| 26 |
+
# Extract original high frequency
|
| 27 |
+
lf_orig = cv2.GaussianBlur(orig_f, (0, 0), sigmaX=3.0)
|
| 28 |
+
hf_orig = orig_f - lf_orig
|
| 29 |
+
|
| 30 |
+
# Inject onto restored image inside skin mask
|
| 31 |
+
blended = rest_f + hf_orig * weight
|
| 32 |
+
blended = np.clip(blended, 0, 255).astype(np.uint8)
|
| 33 |
+
|
| 34 |
+
# Mask blend
|
| 35 |
+
mask_3ch = (skin_mask.astype(np.float32) / 255.0)[:, :, None]
|
| 36 |
+
result = np.uint8(restored_crop * (1 - mask_3ch) + blended * mask_3ch)
|
| 37 |
+
return result
|
| 38 |
+
```
|
.agents/skills/guided-filter-detail-enhancement/SKILL.md
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: guided-filter-detail-enhancement
|
| 3 |
+
description: Guided image filtering, edge-preserving smoothing, detail layer amplification, and halo artifact suppression for ultra-sharp facial details without harsh noise.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Guided Filter Detail Enhancement Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Guided Filtering performs edge-preserving smoothing using a guidance image. By subtracting the guided-filtered base layer from the original image, we extract a pure edge-detail layer that can be amplified without causing ringing or halo artifacts typical of standard unsharp masking.
|
| 10 |
+
|
| 11 |
+
## Code Example
|
| 12 |
+
```python
|
| 13 |
+
import cv2
|
| 14 |
+
import numpy as np
|
| 15 |
+
|
| 16 |
+
def guided_detail_enhance(img, radius=4, eps=0.01, scale=1.4):
|
| 17 |
+
# Guided filter edge-preserving smooth
|
| 18 |
+
guided = cv2.ximgproc.guidedFilter(guide=img, src=img, radius=radius, eps=eps)
|
| 19 |
+
detail = img.astype(np.float32) - guided.astype(np.float32)
|
| 20 |
+
enhanced = img.astype(np.float32) + detail * (scale - 1.0)
|
| 21 |
+
return np.clip(enhanced, 0, 255).astype(np.uint8)
|
| 22 |
+
```
|
.agents/skills/hdr-tone-mapping-exposure/SKILL.md
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: hdr-tone-mapping-exposure
|
| 3 |
+
description: High Dynamic Range tone mapping, shadow detail enhancement, highlight clipping control, gamma adjustments, and local tone mapping.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# HDR Tone Mapping & Exposure Recovery Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Under-exposed shadows and over-exposed highlights lose texture details. Local tone mapping using Durand / Mantiuk tone mapping curves brings out hidden details in portrait shadows and backgrounds.
|
| 10 |
+
|
| 11 |
+
## Code Example
|
| 12 |
+
```python
|
| 13 |
+
import cv2
|
| 14 |
+
import numpy as np
|
| 15 |
+
|
| 16 |
+
def recover_shadows_and_highlights(img_bgr, gamma=1.1, shadow_boost=1.2):
|
| 17 |
+
# Convert to float 0..1
|
| 18 |
+
img_f = img_bgr.astype(np.float32) / 255.0
|
| 19 |
+
|
| 20 |
+
# Shadow mask based on luminance
|
| 21 |
+
gray = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2GRAY).astype(np.float32) / 255.0
|
| 22 |
+
shadow_mask = np.power(1.0 - gray, 2.0)[:, :, None]
|
| 23 |
+
|
| 24 |
+
# Apply gamma & shadow boost
|
| 25 |
+
corrected = np.power(img_f, 1.0 / gamma)
|
| 26 |
+
boosted = corrected * (1.0 + (shadow_boost - 1.0) * shadow_mask)
|
| 27 |
+
|
| 28 |
+
return np.clip(boosted * 255.0, 0, 255).astype(np.uint8)
|
| 29 |
+
```
|
.agents/skills/huggingface-space-docker-deploy/SKILL.md
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: huggingface-space-docker-deploy
|
| 3 |
+
description: Packaging AI web apps into lightweight Docker containers for HuggingFace Spaces deployment, managing CPU thread limits, memory leaks, and model downloading caches.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# HuggingFace Space Docker Deployment & Optimization Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Deploying AI web applications to HuggingFace Spaces via Docker SDK allows full environment control, CPU PyTorch optimization, custom system packages (`libgl1-mesa-glx`), and automated runtime weight downloading.
|
| 10 |
+
|
| 11 |
+
## Standard Dockerfile Template
|
| 12 |
+
```dockerfile
|
| 13 |
+
FROM python:3.11-slim
|
| 14 |
+
|
| 15 |
+
WORKDIR /app
|
| 16 |
+
RUN apt-get update && apt-get install -y git libgl1-mesa-glx libglib2.0-0 && rm -rf /var/lib/apt/lists/*
|
| 17 |
+
|
| 18 |
+
COPY requirements.txt .
|
| 19 |
+
RUN pip install --no-cache-dir -r requirements.txt
|
| 20 |
+
|
| 21 |
+
COPY . .
|
| 22 |
+
|
| 23 |
+
EXPOSE 7860
|
| 24 |
+
CMD ["streamlit", "run", "app.py", "--server.port=7860", "--server.address=0.0.0.0", "--server.headless=true"]
|
| 25 |
+
```
|
.agents/skills/image-quality-metrics-eval/SKILL.md
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: image-quality-metrics-eval
|
| 3 |
+
description: Quantitative and qualitative image evaluation using PSNR, SSIM, LPIPS perceptual distance, FID distribution score, NIQE no-reference quality, and ArcFace identity metrics.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Image Quality Metrics & Evaluation Suite Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Accurate evaluation of image restoration quality requires both full-reference metrics (comparing restored against ground truth) and no-reference quality metrics.
|
| 10 |
+
|
| 11 |
+
## Supported Metrics
|
| 12 |
+
1. **PSNR (Peak Signal-to-Noise Ratio)**: Measures pixel reconstruction fidelity ($dB$). Higher is better ($>28 ext{dB}$).
|
| 13 |
+
2. **SSIM (Structural Similarity Index)**: Measures structural/luminance closeness ($0..1$). Higher is better ($>0.85$).
|
| 14 |
+
3. **LPIPS (Learned Perceptual Image Patch Similarity)**: Measures perceptual distance via VGG/AlexNet features. Lower is better ($<0.15$).
|
| 15 |
+
4. **NIQE (Natural Image Quality Evaluator)**: No-reference naturalness score. Lower is better ($<4.5$).
|
| 16 |
+
5. **ArcFace Cosine Identity**: Facial identity preservation score ($0..1$). Higher is better ($>0.70$).
|
.agents/skills/lab-clahe-color-grading/SKILL.md
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: lab-clahe-color-grading
|
| 3 |
+
description: LAB color space transformation, Contrast Limited Adaptive Histogram Equalization (CLAHE) on L-channel, and skin-tone preserving color enhancement.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# LAB CLAHE & Dynamic Color Grading Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Restoring degraded images often leaves colors washed out or unevenly exposed. CLAHE (Contrast Limited Adaptive Histogram Equalization) applied in the LAB color space boosts local contrast and shadow detail without distorting human skin tones.
|
| 10 |
+
|
| 11 |
+
## Key Concepts
|
| 12 |
+
1. **LAB Color Space**: Separates Luminance ($L$) from color channels ($a$ for red-green, $b$ for yellow-blue).
|
| 13 |
+
2. **L-Channel CLAHE**: Enhances contrast locally on $L$ only, avoiding color shifts.
|
| 14 |
+
3. **Clip Limit & Tile Size**: Use `clipLimit=1.5` to `2.0` and `tileGridSize=(8,8)` for realistic enhancements.
|
| 15 |
+
|
| 16 |
+
## Implementation
|
| 17 |
+
```python
|
| 18 |
+
import cv2
|
| 19 |
+
|
| 20 |
+
def apply_lab_clahe(img_bgr, clip_limit=1.8, tile_size=(8,8)):
|
| 21 |
+
lab = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2LAB)
|
| 22 |
+
l, a, b = cv2.split(lab)
|
| 23 |
+
|
| 24 |
+
clahe = cv2.createCLAHE(clipLimit=clip_limit, tileGridSize=tile_size)
|
| 25 |
+
cl = clahe.apply(l)
|
| 26 |
+
|
| 27 |
+
merged = cv2.merge((cl, a, b))
|
| 28 |
+
return cv2.cvtColor(merged, cv2.COLOR_LAB2BGR)
|
| 29 |
+
```
|
.agents/skills/lmdb-dataset-binary-storage/SKILL.md
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: lmdb-dataset-binary-storage
|
| 3 |
+
description: Converting loose image folders to Lightning Memory-Mapped Database (LMDB) binary datasets for ultra-fast sequential training I/O and zero disk fragmentation.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# LMDB Dataset Binary Storage & High-Speed I/O Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Reading thousands of small loose image files during neural network training causes severe disk I/O bottlenecks and OS page cache thrashing. LMDB maps binary image buffers directly into virtual memory for instant zero-copy reads.
|
| 10 |
+
|
| 11 |
+
## Build LMDB Script Pattern
|
| 12 |
+
```python
|
| 13 |
+
import lmdb
|
| 14 |
+
import cv2
|
| 15 |
+
|
| 16 |
+
def build_lmdb(image_paths, lmdb_path):
|
| 17 |
+
env = lmdb.open(lmdb_path, map_size=1099511627776) # 1TB max allocation
|
| 18 |
+
with env.begin(write=True) as txn:
|
| 19 |
+
for idx, path in enumerate(image_paths):
|
| 20 |
+
with open(path, 'rb') as f:
|
| 21 |
+
img_bin = f.read()
|
| 22 |
+
key = f"{idx:08d}".encode('ascii')
|
| 23 |
+
txn.put(key, img_bin)
|
| 24 |
+
env.close()
|
| 25 |
+
```
|
.agents/skills/noise-estimation-denoising/SKILL.md
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: noise-estimation-denoising
|
| 3 |
+
description: Non-Local Means (NLM) denoising, wavelet thresholding, and bilateral filtering for noise estimation and suppression prior to super-resolution.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Noise Estimation & Pre-Denoising Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
High-frequency noise in input images gets amplified during AI super-resolution and face restoration. Pre-denoising with Fast Non-Local Means or Bilateral filtering smooths uniform noise while preserving structural edges.
|
| 10 |
+
|
| 11 |
+
## Key Methods
|
| 12 |
+
1. **Noise Estimation**: Compute standard deviation of Laplacian response ($\sigma_n = ext{std}(
|
| 13 |
+
abla^2 I)$).
|
| 14 |
+
2. **Fast NLM Denoising**: `cv2.fastNlMeansDenoisingColored(img, None, h=3, hColor=3, templateWindowSize=7, searchWindowSize=21)`.
|
| 15 |
+
3. **Bilateral Filter**: `cv2.bilateralFilter(img, d=5, sigmaColor=25, sigmaSpace=25)`.
|
.agents/skills/onnx-static-int8-quantization/SKILL.md
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: onnx-static-int8-quantization
|
| 3 |
+
description: Converting PyTorch models to ONNX static INT8 format with calibration data readers, preserving accuracy while achieving 3x-4x CPU speedup.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# ONNX Static INT8 Quantization & Calibration Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Static INT8 quantization uses a calibration dataset to determine exact tensor activation ranges ($[\min, \max]$). This results in faster execution and lower memory bandwidth on CPU compared to dynamic quantization.
|
| 10 |
+
|
| 11 |
+
## Key Concepts
|
| 12 |
+
1. **Calibration Data Reader**: Implements `CalibrationDataReader` class providing 50-100 real face inputs.
|
| 13 |
+
2. **Quantization Mode**: `QuantFormat.QDQ` (Quantize-Dequantize) with `QuantType.QInt8`.
|
| 14 |
+
3. **Execution Provider**: CPUExecutionProvider optimized for AVX2/AVX512.
|
| 15 |
+
|
| 16 |
+
## Implementation Code
|
| 17 |
+
```python
|
| 18 |
+
from onnxruntime.quantization import quantize_static, CalibrationDataReader, QuantFormat, QuantType
|
| 19 |
+
|
| 20 |
+
class FaceDataReader(CalibrationDataReader):
|
| 21 |
+
def __init__(self, sample_inputs):
|
| 22 |
+
self.enum_data = iter([{"input": x} for x in sample_inputs])
|
| 23 |
+
def get_next(self):
|
| 24 |
+
return next(self.enum_data, None)
|
| 25 |
+
|
| 26 |
+
def quantize_onnx_static(in_onnx, out_onnx, sample_inputs):
|
| 27 |
+
reader = FaceDataReader(sample_inputs)
|
| 28 |
+
quantize_static(
|
| 29 |
+
in_onnx, out_onnx, reader,
|
| 30 |
+
quant_format=QuantFormat.QDQ,
|
| 31 |
+
weight_type=QuantType.QInt8,
|
| 32 |
+
activation_type=QuantType.QInt8
|
| 33 |
+
)
|
| 34 |
+
```
|
.agents/skills/streamlit-side-by-side-visualizer/SKILL.md
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: streamlit-side-by-side-visualizer
|
| 3 |
+
description: Designing responsive Streamlit visualizer widgets, interactive split-screen image slider comparison components, pixel zoom inspectors, and performance metrics.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Streamlit Side-by-Side Visualizer & UI Component Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Presenting image enhancement results effectively requires interactive UI components like side-by-side split screens, before/after image sliders, and detailed metric breakdown cards (processing time, resolution boost, face count).
|
| 10 |
+
|
| 11 |
+
## Recommended Patterns
|
| 12 |
+
1. **Interactive Image Comparison Widget**: Use custom HTML/JS or Streamlit image comparison components.
|
| 13 |
+
2. **Dynamic Zoom Inspector**: Allow users to inspect $100\%$ face crop regions side-by-side.
|
| 14 |
+
3. **Download Options**: Provide high-speed PNG/JPEG download buttons with custom file naming.
|
.agents/skills/streamlit-thread-state-guidelines/SKILL.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
---
|
| 2 |
name: streamlit-thread-state-guidelines
|
| 3 |
description: Architectural rules and patterns for Streamlit background threading, thread-safe queue IPC, cached function guards, session_state initialization, and progress polling in AI web applications.
|
| 4 |
---
|
|
|
|
| 1 |
+
---
|
| 2 |
name: streamlit-thread-state-guidelines
|
| 3 |
description: Architectural rules and patterns for Streamlit background threading, thread-safe queue IPC, cached function guards, session_state initialization, and progress polling in AI web applications.
|
| 4 |
---
|
.agents/skills/super-resolution-tile-stitching/SKILL.md
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: super-resolution-tile-stitching
|
| 3 |
+
description: Tile-based image processing for ultra-high-resolution images under strict memory limits, including tile padding, overlap cosine blending, and seam removal.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Super-Resolution Tile Processing & Stitching Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
High-resolution images (4K, 8K) exceed CPU/GPU RAM limits when passed whole through deep learning models. Tiling breaks images into overlapping patches, processes each patch independently, and stitches them with smooth cosine blending to eliminate visible seam borders.
|
| 10 |
+
|
| 11 |
+
## Key Concepts
|
| 12 |
+
1. **Tile Size & Overlap**: e.g., tile size $400 imes 400$, overlap $40 ext{px}$.
|
| 13 |
+
2. **Padding**: Mirror pad outer borders to handle edge tiles seamlessly.
|
| 14 |
+
3. **Cosine Weight Mask**: Generate 2D Gaussian/cosine weight maps so patch overlap areas smoothly transition.
|
| 15 |
+
|
| 16 |
+
## Implementation Pattern
|
| 17 |
+
```python
|
| 18 |
+
import numpy as np
|
| 19 |
+
|
| 20 |
+
def generate_tile_weights(tile_h, tile_w, pad):
|
| 21 |
+
# Smooth linear/cosine weight ramp at borders
|
| 22 |
+
w_x = np.ones(tile_w, dtype=np.float32)
|
| 23 |
+
w_y = np.ones(tile_h, dtype=np.float32)
|
| 24 |
+
w_x[:pad] = np.linspace(0, 1, pad)
|
| 25 |
+
w_x[-pad:] = np.linspace(1, 0, pad)
|
| 26 |
+
w_y[:pad] = np.linspace(0, 1, pad)
|
| 27 |
+
w_y[-pad:] = np.linspace(1, 0, pad)
|
| 28 |
+
return np.outer(w_y, w_x)[:, :, None]
|
| 29 |
+
```
|
.agents/skills/unsharp-masking-deconvolution/SKILL.md
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: unsharp-masking-deconvolution
|
| 3 |
+
description: Dynamic unsharp masking (USM), Richardson-Lucy non-blind deconvolution, and high-pass frequency boosting for crisp edge recovery.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Unsharp Masking & Deconvolution Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
Restoring high-frequency details requires carefully tuned sharpening. Standard USM uses a Gaussian blur kernel to isolate high frequencies, while Richardson-Lucy deconvolution reverses point spread function (PSF) blurring.
|
| 10 |
+
|
| 11 |
+
## Formula & Code
|
| 12 |
+
USM Formula: $I_{sharp} = I + lpha (I - G_\sigma * I)$
|
| 13 |
+
|
| 14 |
+
```python
|
| 15 |
+
import cv2
|
| 16 |
+
import numpy as np
|
| 17 |
+
|
| 18 |
+
def adaptive_unsharp_mask(img, amount=0.5, radius=1.0, threshold=2):
|
| 19 |
+
blurred = cv2.GaussianBlur(img, (0, 0), radius)
|
| 20 |
+
diff = img.astype(np.float32) - blurred.astype(np.float32)
|
| 21 |
+
mask = np.abs(diff) >= threshold
|
| 22 |
+
sharpened = img.astype(np.float32) + amount * diff
|
| 23 |
+
result = np.where(mask, sharpened, img.astype(np.float32))
|
| 24 |
+
return np.clip(result, 0, 255).astype(np.uint8)
|
| 25 |
+
```
|
.agents/skills/vqgan-codebook-fine-tuning/SKILL.md
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: vqgan-codebook-fine-tuning
|
| 3 |
+
description: Fine-tuning VQGAN discrete codebooks, discrete code dictionary index lookup, stage I VQ autoencoders, and codebook matching for photo & illustration styles.
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# VQGAN Codebook Fine-Tuning & Dictionary Matching Skill
|
| 7 |
+
|
| 8 |
+
## Overview
|
| 9 |
+
CodeFormer relies on a VQGAN codebook (Stage I) that encapsulates high-quality clean face priors. When restoring domain-specific images (game characters, vintage portraits), fine-tuning codebook vectors or adjusting codebook lookup weights improves naturalness and reduces vector quantization quantization loss.
|
| 10 |
+
|
| 11 |
+
## Key Components
|
| 12 |
+
1. **Codebook Size**: $N=1024$ codebook vectors of dimension $d=512$.
|
| 13 |
+
2. **Quantization Loss**: Commitment loss + Codebook update loss.
|
| 14 |
+
3. **Cross-Entropy Code Prediction Transformer**: Stage II maps low-quality features to discrete codebook indices.
|
models/CodeFormer/options/cf_runtime_szvghq3h.yml
ADDED
|
@@ -0,0 +1,166 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
name: CodeFormer_stage3_custom
|
| 2 |
+
model_type: CodeFormerJointModel
|
| 3 |
+
num_gpu: 0
|
| 4 |
+
manual_seed: 0
|
| 5 |
+
datasets:
|
| 6 |
+
train:
|
| 7 |
+
name: CustomDataset
|
| 8 |
+
type: FFHQBlindJointDataset
|
| 9 |
+
dataroot_gt: datasets/ffhq/ffhq_512
|
| 10 |
+
filename_tmpl: '{}'
|
| 11 |
+
io_backend:
|
| 12 |
+
type: disk
|
| 13 |
+
in_size: 512
|
| 14 |
+
gt_size: 512
|
| 15 |
+
mean:
|
| 16 |
+
- 0.5
|
| 17 |
+
- 0.5
|
| 18 |
+
- 0.5
|
| 19 |
+
std:
|
| 20 |
+
- 0.5
|
| 21 |
+
- 0.5
|
| 22 |
+
- 0.5
|
| 23 |
+
use_hflip: true
|
| 24 |
+
use_corrupt: true
|
| 25 |
+
blur_kernel_size: 41
|
| 26 |
+
use_motion_kernel: true
|
| 27 |
+
motion_kernel_prob: 0.15
|
| 28 |
+
kernel_list:
|
| 29 |
+
- iso
|
| 30 |
+
- aniso
|
| 31 |
+
kernel_prob:
|
| 32 |
+
- 0.5
|
| 33 |
+
- 0.5
|
| 34 |
+
blur_sigma:
|
| 35 |
+
- 0.1
|
| 36 |
+
- 10.0
|
| 37 |
+
downsample_range:
|
| 38 |
+
- 1.0
|
| 39 |
+
- 20.0
|
| 40 |
+
noise_range:
|
| 41 |
+
- 0.0
|
| 42 |
+
- 30.0
|
| 43 |
+
jpeg_range:
|
| 44 |
+
- 10
|
| 45 |
+
- 70
|
| 46 |
+
blur_sigma_large:
|
| 47 |
+
- 1.0
|
| 48 |
+
- 15.0
|
| 49 |
+
downsample_range_large:
|
| 50 |
+
- 4.0
|
| 51 |
+
- 30.0
|
| 52 |
+
noise_range_large:
|
| 53 |
+
- 0.0
|
| 54 |
+
- 30.0
|
| 55 |
+
jpeg_range_large:
|
| 56 |
+
- 5
|
| 57 |
+
- 50
|
| 58 |
+
latent_gt_path: null
|
| 59 |
+
num_worker_per_gpu: 0
|
| 60 |
+
batch_size_per_gpu: 1
|
| 61 |
+
dataset_enlarge_ratio: 5
|
| 62 |
+
prefetch_mode: null
|
| 63 |
+
network_g:
|
| 64 |
+
type: CodeFormer
|
| 65 |
+
dim_embd: 512
|
| 66 |
+
n_head: 8
|
| 67 |
+
n_layers: 9
|
| 68 |
+
codebook_size: 1024
|
| 69 |
+
connect_list:
|
| 70 |
+
- '32'
|
| 71 |
+
- '64'
|
| 72 |
+
- '128'
|
| 73 |
+
- '256'
|
| 74 |
+
fix_modules:
|
| 75 |
+
- quantize
|
| 76 |
+
- generator
|
| 77 |
+
network_vqgan:
|
| 78 |
+
type: VQAutoEncoder
|
| 79 |
+
img_size: 512
|
| 80 |
+
nf: 64
|
| 81 |
+
ch_mult:
|
| 82 |
+
- 1
|
| 83 |
+
- 2
|
| 84 |
+
- 2
|
| 85 |
+
- 4
|
| 86 |
+
- 4
|
| 87 |
+
- 8
|
| 88 |
+
quantizer: nearest
|
| 89 |
+
codebook_size: 1024
|
| 90 |
+
network_d:
|
| 91 |
+
type: VQGANDiscriminator
|
| 92 |
+
nc: 3
|
| 93 |
+
ndf: 64
|
| 94 |
+
n_layers: 4
|
| 95 |
+
path:
|
| 96 |
+
pretrain_network_g: null
|
| 97 |
+
param_key_g: params_ema
|
| 98 |
+
strict_load_g: false
|
| 99 |
+
pretrain_network_d: null
|
| 100 |
+
resume_state: D:\.gemini-scratch\custom-ai-enhancer\models\CodeFormer\experiments\20260708_201102_CodeFormer_stage3_custom\training_states\2150.state
|
| 101 |
+
train:
|
| 102 |
+
use_hq_feat_loss: true
|
| 103 |
+
feat_loss_weight: 1.0
|
| 104 |
+
cross_entropy_loss: true
|
| 105 |
+
entropy_loss_weight: 0.5
|
| 106 |
+
identity_loss_weight: 0.5
|
| 107 |
+
scale_adaptive_gan_weight: 0.1
|
| 108 |
+
optim_g:
|
| 109 |
+
type: Adam
|
| 110 |
+
lr: 5.0e-05
|
| 111 |
+
weight_decay: 0
|
| 112 |
+
betas:
|
| 113 |
+
- 0.9
|
| 114 |
+
- 0.99
|
| 115 |
+
optim_d:
|
| 116 |
+
type: Adam
|
| 117 |
+
lr: 5.0e-05
|
| 118 |
+
weight_decay: 0
|
| 119 |
+
betas:
|
| 120 |
+
- 0.9
|
| 121 |
+
- 0.99
|
| 122 |
+
scheduler:
|
| 123 |
+
type: CosineAnnealingRestartLR
|
| 124 |
+
periods:
|
| 125 |
+
- 20000
|
| 126 |
+
restart_weights:
|
| 127 |
+
- 1
|
| 128 |
+
eta_min: 5.0e-06
|
| 129 |
+
total_iter: 20000
|
| 130 |
+
warmup_iter: -1
|
| 131 |
+
ema_decay: 0.997
|
| 132 |
+
pixel_opt:
|
| 133 |
+
type: L1Loss
|
| 134 |
+
loss_weight: 1.0
|
| 135 |
+
reduction: mean
|
| 136 |
+
perceptual_opt:
|
| 137 |
+
type: LPIPSLoss
|
| 138 |
+
loss_weight: 1.0
|
| 139 |
+
use_input_norm: true
|
| 140 |
+
range_norm: true
|
| 141 |
+
gan_opt:
|
| 142 |
+
type: GANLoss
|
| 143 |
+
gan_type: hinge
|
| 144 |
+
loss_weight: 1.0
|
| 145 |
+
use_adaptive_weight: true
|
| 146 |
+
net_g_start_iter: 0
|
| 147 |
+
net_d_iters: 1
|
| 148 |
+
net_d_start_iter: 0
|
| 149 |
+
manual_seed: 0
|
| 150 |
+
val:
|
| 151 |
+
val_freq: 500
|
| 152 |
+
save_img: true
|
| 153 |
+
metrics:
|
| 154 |
+
psnr:
|
| 155 |
+
type: calculate_psnr
|
| 156 |
+
crop_border: 4
|
| 157 |
+
test_y_channel: false
|
| 158 |
+
logger:
|
| 159 |
+
print_freq: 1
|
| 160 |
+
save_checkpoint_freq: 50
|
| 161 |
+
use_tb_logger: false
|
| 162 |
+
wandb:
|
| 163 |
+
project: null
|
| 164 |
+
find_unused_parameters: true
|
| 165 |
+
dist_params: null
|
| 166 |
+
dist: false
|
models/CodeFormer/options/cf_runtime_vmvjxo41.yml
ADDED
|
@@ -0,0 +1,166 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
name: CodeFormer_stage3_custom
|
| 2 |
+
model_type: CodeFormerJointModel
|
| 3 |
+
num_gpu: 0
|
| 4 |
+
manual_seed: 0
|
| 5 |
+
datasets:
|
| 6 |
+
train:
|
| 7 |
+
name: CustomDataset
|
| 8 |
+
type: FFHQBlindJointDataset
|
| 9 |
+
dataroot_gt: datasets/ffhq/ffhq_512
|
| 10 |
+
filename_tmpl: '{}'
|
| 11 |
+
io_backend:
|
| 12 |
+
type: disk
|
| 13 |
+
in_size: 512
|
| 14 |
+
gt_size: 512
|
| 15 |
+
mean:
|
| 16 |
+
- 0.5
|
| 17 |
+
- 0.5
|
| 18 |
+
- 0.5
|
| 19 |
+
std:
|
| 20 |
+
- 0.5
|
| 21 |
+
- 0.5
|
| 22 |
+
- 0.5
|
| 23 |
+
use_hflip: true
|
| 24 |
+
use_corrupt: true
|
| 25 |
+
blur_kernel_size: 41
|
| 26 |
+
use_motion_kernel: true
|
| 27 |
+
motion_kernel_prob: 0.15
|
| 28 |
+
kernel_list:
|
| 29 |
+
- iso
|
| 30 |
+
- aniso
|
| 31 |
+
kernel_prob:
|
| 32 |
+
- 0.5
|
| 33 |
+
- 0.5
|
| 34 |
+
blur_sigma:
|
| 35 |
+
- 0.1
|
| 36 |
+
- 10.0
|
| 37 |
+
downsample_range:
|
| 38 |
+
- 1.0
|
| 39 |
+
- 20.0
|
| 40 |
+
noise_range:
|
| 41 |
+
- 0.0
|
| 42 |
+
- 30.0
|
| 43 |
+
jpeg_range:
|
| 44 |
+
- 10
|
| 45 |
+
- 70
|
| 46 |
+
blur_sigma_large:
|
| 47 |
+
- 1.0
|
| 48 |
+
- 15.0
|
| 49 |
+
downsample_range_large:
|
| 50 |
+
- 4.0
|
| 51 |
+
- 30.0
|
| 52 |
+
noise_range_large:
|
| 53 |
+
- 0.0
|
| 54 |
+
- 30.0
|
| 55 |
+
jpeg_range_large:
|
| 56 |
+
- 5
|
| 57 |
+
- 50
|
| 58 |
+
latent_gt_path: null
|
| 59 |
+
num_worker_per_gpu: 0
|
| 60 |
+
batch_size_per_gpu: 1
|
| 61 |
+
dataset_enlarge_ratio: 5
|
| 62 |
+
prefetch_mode: null
|
| 63 |
+
network_g:
|
| 64 |
+
type: CodeFormer
|
| 65 |
+
dim_embd: 512
|
| 66 |
+
n_head: 8
|
| 67 |
+
n_layers: 9
|
| 68 |
+
codebook_size: 1024
|
| 69 |
+
connect_list:
|
| 70 |
+
- '32'
|
| 71 |
+
- '64'
|
| 72 |
+
- '128'
|
| 73 |
+
- '256'
|
| 74 |
+
fix_modules:
|
| 75 |
+
- quantize
|
| 76 |
+
- generator
|
| 77 |
+
network_vqgan:
|
| 78 |
+
type: VQAutoEncoder
|
| 79 |
+
img_size: 512
|
| 80 |
+
nf: 64
|
| 81 |
+
ch_mult:
|
| 82 |
+
- 1
|
| 83 |
+
- 2
|
| 84 |
+
- 2
|
| 85 |
+
- 4
|
| 86 |
+
- 4
|
| 87 |
+
- 8
|
| 88 |
+
quantizer: nearest
|
| 89 |
+
codebook_size: 1024
|
| 90 |
+
network_d:
|
| 91 |
+
type: VQGANDiscriminator
|
| 92 |
+
nc: 3
|
| 93 |
+
ndf: 64
|
| 94 |
+
n_layers: 4
|
| 95 |
+
path:
|
| 96 |
+
pretrain_network_g: null
|
| 97 |
+
param_key_g: params_ema
|
| 98 |
+
strict_load_g: false
|
| 99 |
+
pretrain_network_d: null
|
| 100 |
+
resume_state: D:\.gemini-scratch\custom-ai-enhancer\models\CodeFormer\experiments\20260708_201102_CodeFormer_stage3_custom\training_states\2150.state
|
| 101 |
+
train:
|
| 102 |
+
use_hq_feat_loss: true
|
| 103 |
+
feat_loss_weight: 1.0
|
| 104 |
+
cross_entropy_loss: true
|
| 105 |
+
entropy_loss_weight: 0.5
|
| 106 |
+
identity_loss_weight: 0.5
|
| 107 |
+
scale_adaptive_gan_weight: 0.1
|
| 108 |
+
optim_g:
|
| 109 |
+
type: Adam
|
| 110 |
+
lr: 5.0e-05
|
| 111 |
+
weight_decay: 0
|
| 112 |
+
betas:
|
| 113 |
+
- 0.9
|
| 114 |
+
- 0.99
|
| 115 |
+
optim_d:
|
| 116 |
+
type: Adam
|
| 117 |
+
lr: 5.0e-05
|
| 118 |
+
weight_decay: 0
|
| 119 |
+
betas:
|
| 120 |
+
- 0.9
|
| 121 |
+
- 0.99
|
| 122 |
+
scheduler:
|
| 123 |
+
type: CosineAnnealingRestartLR
|
| 124 |
+
periods:
|
| 125 |
+
- 20000
|
| 126 |
+
restart_weights:
|
| 127 |
+
- 1
|
| 128 |
+
eta_min: 5.0e-06
|
| 129 |
+
total_iter: 20000
|
| 130 |
+
warmup_iter: -1
|
| 131 |
+
ema_decay: 0.997
|
| 132 |
+
pixel_opt:
|
| 133 |
+
type: L1Loss
|
| 134 |
+
loss_weight: 1.0
|
| 135 |
+
reduction: mean
|
| 136 |
+
perceptual_opt:
|
| 137 |
+
type: LPIPSLoss
|
| 138 |
+
loss_weight: 1.0
|
| 139 |
+
use_input_norm: true
|
| 140 |
+
range_norm: true
|
| 141 |
+
gan_opt:
|
| 142 |
+
type: GANLoss
|
| 143 |
+
gan_type: hinge
|
| 144 |
+
loss_weight: 1.0
|
| 145 |
+
use_adaptive_weight: true
|
| 146 |
+
net_g_start_iter: 0
|
| 147 |
+
net_d_iters: 1
|
| 148 |
+
net_d_start_iter: 0
|
| 149 |
+
manual_seed: 0
|
| 150 |
+
val:
|
| 151 |
+
val_freq: 500
|
| 152 |
+
save_img: true
|
| 153 |
+
metrics:
|
| 154 |
+
psnr:
|
| 155 |
+
type: calculate_psnr
|
| 156 |
+
crop_border: 4
|
| 157 |
+
test_y_channel: false
|
| 158 |
+
logger:
|
| 159 |
+
print_freq: 1
|
| 160 |
+
save_checkpoint_freq: 50
|
| 161 |
+
use_tb_logger: false
|
| 162 |
+
wandb:
|
| 163 |
+
project: null
|
| 164 |
+
find_unused_parameters: true
|
| 165 |
+
dist_params: null
|
| 166 |
+
dist: false
|
wink_enhancer.py
CHANGED
|
@@ -35,17 +35,20 @@ class WinkQualityEnhancer:
|
|
| 35 |
grain_layer = (high_freq * grain_amount).clip(-128, 127)
|
| 36 |
|
| 37 |
if skin_mask is not None:
|
| 38 |
-
|
| 39 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
else:
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
# Skin category in facexlib parse mask is index 1
|
| 44 |
-
skin_binary = (skin_mask_resized == 1).astype(np.float32)
|
| 45 |
-
# Smooth mask edge
|
| 46 |
-
skin_binary = cv2.GaussianBlur(skin_binary, (5, 5), 0)[:, :, np.newaxis]
|
| 47 |
-
|
| 48 |
-
blended = restored_face.astype(np.float32) + grain_layer * skin_binary
|
| 49 |
else:
|
| 50 |
blended = restored_face.astype(np.float32) + grain_layer
|
| 51 |
|
|
@@ -64,11 +67,17 @@ class WinkQualityEnhancer:
|
|
| 64 |
return cv2.addWeighted(face_img, 1.15, blur, -0.15, 0)
|
| 65 |
|
| 66 |
try:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
h, w = face_img.shape[:2]
|
| 68 |
-
if
|
| 69 |
-
parse_mask_res = cv2.resize(
|
| 70 |
else:
|
| 71 |
-
parse_mask_res =
|
|
|
|
| 72 |
|
| 73 |
# Facial feature mask IDs in facexlib:
|
| 74 |
# 4: Left Eye, 5: Right Eye, 6: Glasses, 11: Upper Lip, 12: Lower Lip, 13: Inner Mouth
|
|
|
|
| 35 |
grain_layer = (high_freq * grain_amount).clip(-128, 127)
|
| 36 |
|
| 37 |
if skin_mask is not None:
|
| 38 |
+
skin_mask_2d = np.squeeze(skin_mask)
|
| 39 |
+
if skin_mask_2d.ndim == 2:
|
| 40 |
+
if skin_mask_2d.shape != restored_face.shape[:2]:
|
| 41 |
+
skin_mask_resized = cv2.resize(skin_mask_2d.astype(np.uint8), (restored_face.shape[1], restored_face.shape[0]), interpolation=cv2.INTER_NEAREST)
|
| 42 |
+
else:
|
| 43 |
+
skin_mask_resized = skin_mask_2d
|
| 44 |
+
|
| 45 |
+
# Skin category in facexlib parse mask is index 1
|
| 46 |
+
skin_binary = (skin_mask_resized == 1).astype(np.float32)
|
| 47 |
+
# Smooth mask edge
|
| 48 |
+
skin_binary = cv2.GaussianBlur(skin_binary, (5, 5), 0)[:, :, np.newaxis]
|
| 49 |
+
blended = restored_face.astype(np.float32) + grain_layer * skin_binary
|
| 50 |
else:
|
| 51 |
+
blended = restored_face.astype(np.float32) + grain_layer
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
else:
|
| 53 |
blended = restored_face.astype(np.float32) + grain_layer
|
| 54 |
|
|
|
|
| 67 |
return cv2.addWeighted(face_img, 1.15, blur, -0.15, 0)
|
| 68 |
|
| 69 |
try:
|
| 70 |
+
parse_mask_2d = np.squeeze(parse_mask)
|
| 71 |
+
if parse_mask_2d.ndim != 2:
|
| 72 |
+
blur = cv2.GaussianBlur(face_img, (0, 0), 2.0)
|
| 73 |
+
return cv2.addWeighted(face_img, 1.15, blur, -0.15, 0)
|
| 74 |
+
|
| 75 |
h, w = face_img.shape[:2]
|
| 76 |
+
if parse_mask_2d.shape[:2] != (h, w):
|
| 77 |
+
parse_mask_res = cv2.resize(parse_mask_2d.astype(np.uint8), (w, h), interpolation=cv2.INTER_NEAREST)
|
| 78 |
else:
|
| 79 |
+
parse_mask_res = parse_mask_2d
|
| 80 |
+
|
| 81 |
|
| 82 |
# Facial feature mask IDs in facexlib:
|
| 83 |
# 4: Left Eye, 5: Right Eye, 6: Glasses, 11: Upper Lip, 12: Lower Lip, 13: Inner Mouth
|