Upload folder using huggingface_hub

README.md

@@ -0,0 +1,115 @@
+---
+license: cc-by-nc-4.0
+base_model: stabilityai/stable-diffusion-3-medium-diffusers
+library_name: peft
+pipeline_tag: image-to-image
+language: en
+tags:
+  - super-resolution
+  - image-super-resolution
+  - extreme-zoom
+  - chain-of-zoom
+  - diffusion
+  - lora
+  - peft
+  - privileged-distillation
+  - faithfulness
+---
+
+# OracleZoom
+
+**Privileged-Latent Distillation for faithful extreme super-resolution.**
+A tiny (7.1M-parameter) LoRA adapter that makes Chain-of-Zoom's recursive super-resolution add *faithful* detail instead of hallucinating, all the way to 256x.
+
+[![GitHub](https://img.shields.io/badge/Code-OPD--Zoom-black?logo=github)](https://github.com/dipta007/OPD-Zoom)
+[![Base](https://img.shields.io/badge/Backbone-OSEDiff%20/%20SD3-blue)](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers)
+[![Method](https://img.shields.io/badge/Method-Chain--of--Zoom-orange)](https://github.com/dipta007/OPD-Zoom)
+[![Paper](https://img.shields.io/badge/Paper-WACV%202027%20(in%20submission)-red)](https://github.com/dipta007/OPD-Zoom)
+[![License](https://img.shields.io/badge/License-CC--BY--NC--4.0-lightgrey)](https://creativecommons.org/licenses/by-nc/4.0/)
+
+## Highlights
+- **Faithful, not just sharp.** At extreme zoom the backbone must *invent* detail; the question is whether it is faithful or hallucinated. This adapter teaches faithfulness.
+- **Holds where baselines collapse.** CLIPIQA **0.71 at 256x** while Chain-of-Zoom (CoZ) and five SOTA SR backbones fall to <=0.58; most faithful of all methods at 4x (LPIPS **0.20** vs CoZ 0.22).
+- **Judged more faithful.** Two cross-family vision-language judges (InternVL + Gemini) prefer this zoom **68-78%** of the time at 64-256x and flag the strongest baseline hallucinating **2-5x more**.
+- **Tiny and drop-in.** A rank-16 LoRA (**7.1M** trainable params) trained on only **1,000** curated 4K images; it drops into CoZ's recursion with no other changes.
+
+## Model Overview
+| Property | Value |
+|---|---|
+| Model type | LoRA adapter (PEFT) for a one-step SR backbone |
+| Backbone (frozen) | OSEDiff on Stable Diffusion 3-medium |
+| Prompt extractor (frozen) | Qwen2.5-VL-3B-Instruct |
+| Trainable params | 7.1M |
+| LoRA | r = 16, alpha = 32, dropout = 0.0 |
+| LoRA targets | `to_q, to_k, to_v, add_q_proj, add_k_proj, add_v_proj` (SD3 transformer) |
+| Training data | 1,000 curated 4K photographs (supervised at 4x only) |
+| Objective | decode-space LPIPS + anchored cycle-consistency - beta_reward * TOPIQ-NR + beta_kl * KL-to-base + EMA |
+| Key weights | beta_reward 0.4, beta_kl 8.0, w_cyc 1.0, lambda_ema 0.1 (EMA decay 0.95) |
+| Recursion at test | 4 steps (4x / 16x / 64x / 256x), 512x512 center crop |
+
+## Method
+Recursive SR (Chain-of-Zoom) reuses a 4x backbone step after step to reach 16x-256x. Each step is **blind**: it sees only a blurred crop of its own previous output and must invent the missing detail, so errors compound and the invention may be hallucinated.
+
+**Privileged-latent distillation (the idea).** A *privileged teacher* is shown the ground-truth high-resolution patch **at training time only** and distills its real detail into the blind student, in **decode space** (a perceptual loss between the student's decoded image and the real patch). Only a small LoRA adapter is trained; the backbone, VAE, and prompter stay frozen.
+
+**A KL leash keeps the deep reward faithful (the safeguard).** Ground truth exists only at 4x. To carry the distilled prior into the deeper recursion, the student chases a differentiable detail reward (TOPIQ-NR) through the real zoom. Left free, such a reward games the metric with a repetitive crosshatch; **leashed** to the deployed backbone by a KL trust region (a latent distance for a one-step map), it sharpens detail without drifting into hallucination.
+
+Full derivation and ablations are in the [OPD-Zoom repo](https://github.com/dipta007/OPD-Zoom).
+
+## Quickstart
+This is the trained artifact of OracleZoom: a PEFT LoRA on the SD3 transformer that OSEDiff uses as the SR backbone inside Chain-of-Zoom. It plugs into the OPD-Zoom pipeline.
+
+```bash
+# 1) get the pipeline
+git clone https://github.com/dipta007/OPD-Zoom && cd OPD-Zoom
+
+# 2) get this adapter
+huggingface-cli download dipta007/OracleZoom --local-dir ckpt/OracleZoom
+
+# 3) run the 4-recursion zoom with the adapter as the student SR LoRA
+python -m opd_zoom.teacher.oracle_infer \
+  --mode student --pld_lora ckpt/OracleZoom \
+  --gt_dir <your_images> --out <out_dir> --rec_num 4
+```
+
+Loading just the adapter with PEFT:
+```python
+from peft import PeftModel
+# `sd3_transformer` is the SD3Transformer2DModel used by the OSEDiff backbone
+model = PeftModel.from_pretrained(sd3_transformer, "dipta007/OracleZoom")
+```
+The adapter is `inference_mode` and merges into the backbone at no added latency; the VLM prompter is unchanged, so per-image inference cost equals Chain-of-Zoom's.
+
+## Results
+Under Chain-of-Zoom's exact protocol on a curated 4K benchmark and six test sets (in-domain 4K, DIV8K, DRealSR, RealSR, FFHQ, Flickr2K):
+
+| Axis | Metric | Ours | CoZ / best baseline |
+|---|---|---|---|
+| Sharpness (no-reference) | CLIPIQA @256x | **0.71** | <= 0.58 |
+| Fidelity @4x (ground truth exists) | LPIPS | **0.20** | 0.22 (CoZ) |
+| Deep faithfulness (MLLM judge, 64-256x) | preferred vs CoZ | **68-78%** | - |
+| Deep faithfulness | hallucination rate vs CoZ | **2-5x lower** | - |
+
+Sharpness is the axis prior methods are built for; the decisive gap is **faithfulness**, verified by full-reference metrics at 4x and by two cross-family MLLM judges plus a blinded human study past 4x.
+
+## Intended Use
+- **In-scope:** research on faithful extreme (recursive) super-resolution; as the SR-backbone adapter inside the Chain-of-Zoom recursion on natural photographs.
+- **Out-of-scope:** a standalone single-shot SR model (it is a drop-in LoRA for the CoZ loop, not a full model); forensic or evidentiary use (detail past 4x is generated, not recovered); real-camera-zoom claims (the benchmark uses synthetic center-crop zoom).
+
+## Training
+Early-stopped on held-out validation at ~epoch 37 (step 9300); best val 0.216. Trained on one 8xH200 node (single GPU trains the adapter). Full config in `train_meta.json` and the [repo](https://github.com/dipta007/OPD-Zoom).
+
+## Citation
+```bibtex
+@inproceedings{dipta2027oraclezoom,
+  title={OracleZoom: Privileged-Latent Distillation for Faithful Extreme Super-Resolution},
+  author={Shubhashis Roy Dipta},
+  year={2027},
+  note={In submission, WACV 2027},
+  url={https://github.com/dipta007/OPD-Zoom}
+}
+```
+Please also cite Chain-of-Zoom and OSEDiff, whose components this builds on.
+
+## License
+Released for **research, non-commercial** use (CC-BY-NC-4.0). This adapter is trained on top of OSEDiff / Stable Diffusion 3 and used with a Qwen2.5-VL prompter inside Chain-of-Zoom; the respective upstream licenses apply to those components.

adapter_config.json

@@ -0,0 +1,41 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "SD3Transformer2DModel",
+    "parent_library": "diffusers.models.transformers.transformer_sd3"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "add_q_proj",
+    "to_v",
+    "add_k_proj",
+    "to_q",
+    "to_k",
+    "add_v_proj"
+  ],
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:612b3ac0c2ec152bf674ccacf05e9d0a964cba9cc7f77d87bd19c55a6876bc86
+size 28350704

train_meta.json

@@ -0,0 +1 @@
+{"best_val": 0.21588097116095012, "best_step": 9300, "best_epoch": 37, "stopped_early": true, "w_deep": 1.0, "deep_factor": 4, "lambda_ema": 0.1, "ema_decay": 0.95, "no_ema": false, "full_grad": true, "beta_reward": 0.4, "reward_metric": "topiq_nr", "beta_kl": 8.0, "lora_on": "transformer", "full_param": false, "lora_rank": 16, "n_train": 1000, "n_val": 2000}