Upload 1112 files

.gitattributes

@@ -33,3 +33,17 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+sam2/assets/model_diagram.png filter=lfs diff=lfs merge=lfs -text
+sam2/assets/sa_v_dataset.jpg filter=lfs diff=lfs merge=lfs -text
+sam2/demo/data/gallery/01_dog.mp4 filter=lfs diff=lfs merge=lfs -text
+sam2/demo/data/gallery/02_cups.mp4 filter=lfs diff=lfs merge=lfs -text
+sam2/demo/data/gallery/03_blocks.mp4 filter=lfs diff=lfs merge=lfs -text
+sam2/demo/data/gallery/04_coffee.mp4 filter=lfs diff=lfs merge=lfs -text
+sam2/demo/data/gallery/05_default_juggle.mp4 filter=lfs diff=lfs merge=lfs -text
+sam2/demo/frontend/public/fonts/Inter-VariableFont_opsz,wght.ttf filter=lfs diff=lfs merge=lfs -text
+sam2/demo/frontend/src/assets/videos/sam2_720px_dark.mp4 filter=lfs diff=lfs merge=lfs -text
+sam2/notebooks/images/cars.jpg filter=lfs diff=lfs merge=lfs -text
+sam2/notebooks/images/groceries.jpg filter=lfs diff=lfs merge=lfs -text
+sam2/notebooks/images/truck.jpg filter=lfs diff=lfs merge=lfs -text
+sam2/notebooks/videos/bedroom.mp4 filter=lfs diff=lfs merge=lfs -text
+sam2/sav_dataset/example/sav_000001.mp4 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,79 @@
+## ⚙️ Dependencies and Installation
+```
+## git clone this repository
+git clone https://huggingface.co/NGain/Medialab
+cd Medialab
+
+# create an environment with python >= 3.8
+conda create -n medialab python=3.8
+conda activate medialab
+pip install -r requirements.txt
+
+# or you can directly install the environment by following instruct
+conda env create -f medialab.yml
+conda activate medialab
+```
+
+## 🚀 Quick Inference
+#### Step 1: Download the pretrained models
+- Download the pretrained SD-2-base models from [HuggingFace](https://huggingface.co/stabilityai/stable-diffusion-2-base)
+- Download the checkpoint, sam2.1_hiera_tiny, ram_swin_large and DAPE models from [GoogleDrive](https://drive.google.com/drive/folders/1Ce0D8R99t-fDQfACLc8SGvf3gzdMnTwT?usp=sharing).
+
+You can put the models into `preset/models`.
+
+#### Step 2: Prepare testing data
+You can put the testing images in the `preset/datasets/test_datasets`.
+
+#### Step 3: Running testing command
+```
+# for wild dataset
+python ./test_seesr_sam.py \
+--pretrained_model_path ./preset/models/stable-diffusion-2-base \
+--prompt '' \
+--seesr_model_path ./preset/models/checkpoint-90000 \
+--ram_ft_path ./preset/models/DAPE.pth \
+--image_path ./preset/datasets/test_datasets/wild \
+--output_dir your_output_dir_path/wild \
+--start_point noise \
+--num_inference_steps 50 \
+--guidance_scale 14 \
+--added_prompt "clean, high-resolution, 8k, ultra-detailed, ultra-realistic" \
+--upscale 1 \
+--process_size 512
+
+# for synthetic dataset
+python ./test_seesr_sam.py \
+--pretrained_model_path ./preset/models/stable-diffusion-2-base \
+--prompt '' \
+--seesr_model_path ./preset/models/checkpoint-90000 \
+--ram_ft_path ./preset/models/DAPE.pth \
+--image_path ./preset/datasets/test_datasets/synthetic \
+--output_dir your_output_dir_path/synthetic \
+--start_point noise \
+--num_inference_steps 50 \
+--guidance_scale 0.9 \
+--upscale 4 \
+--process_size 512
+```
+More details are [here](asserts/hyp.md)
+
+## 🌈 Train 
+
+Will release soon.
+
+## ❤️ Acknowledgments
+This project is based on [diffusers](https://github.com/huggingface/diffusers) and [SeeSR](https://github.com/cswry/SeeSR). Some codes are brought from [PASD](https://github.com/yangxy/PASD), [RAM](https://github.com/xinyu1205/recognize-anything) and [SAM2](https://github.com/facebookresearch/sam2)). Thanks for their awesome works. We also pay tribute to the pioneering work of [StableSR](https://github.com/IceClear/StableSR).
+
+## 📧 Contact
+If you have any questions, please feel free to contact: `ly5825761@gmail.com`
+
+## 🎫 License
+This project and related weights are released under the [Apache 2.0 license](LICENSE).
+
+
+<details>
+<summary>statistics</summary>
+
+![visitors](https://visitor-badge.laobi.icu/badge?page_id=cswry/SeeSR)
+
+</details>

asserts/hyp.md

@@ -0,0 +1,23 @@
+Default inference settings
+```
+python test_seesr.py \
+--pretrained_model_path preset/models/stable-diffusion-2-base \
+--prompt '' \
+--seesr_model_path preset/models/seesr \
+--ram_ft_path preset/models/DAPE.pth \
+--image_path preset/datasets/test_datasets \
+--output_dir preset/datasets/output \
+--start_point lr \
+--num_inference_steps 50 \
+--guidance_scale 5.5 \
+--process_size 512 
+```
+
+The default settings are optimized for the best result. However, the behavior of the SeeSR can be customized
+- Trade-offs between the **fidelity** and **perception**  
+  - `--num_inference_steps` Using more sampling steps in `Real-world SR` tasks is not a purely beneficial choice. While it improves the perception quality, it can also reduce fidelity quality as it generates more. Considering the trade-offs between fidelity and perception, as well as the inference time cost, we set the default value to `50`. However, you can make appropriate adjustments based on your specific needs.
+  - `--guidance_scale` A higher value means unleashing more generation capacity of SD, which improves perception quality but decreases fidelity quality. We set the default value to `5.5`, you can make appropriate adjustments based on your specific needs.
+  - `--process_size` The inference script resizes input images to the `process_size`, and then resizes the prediction back to the original resolution after process. We found that increasing the processing size (e.g. 768) improves fidelity but decreases perception. We set the default value to `512`, consistent with the training size of the pre-trained SD model. You can make appropriate adjustments based on your specific needs.
+
+- User-specified mode
+  - `--prompt` SeeSR utilizes DAPE to automatically extract tag prompts from LR images, but it is not the most perfect approach. You can try manually specifying appropriate tag prompts to further enhance the quality of the results.

asserts/turbo.md

@@ -0,0 +1,5 @@
+#### Inference with 2 steps:
+[<img src="../figs/turbo_steps02_frog.png" height="350px"/>](https://imgsli.com/MjQ2ODA0) [<img src="../figs/turbo_steps02_building.png" height="350px"/>](https://imgsli.com/MjQ2ODA2)
+
+#### Inference with 4 steps:
+[<img src="../figs/turbo_steps04_frog.png" height="350px"/>](https://imgsli.com/MjQ2ODEw) [<img src="../figs/turbo_steps04_building.png" height="350px"/>](https://imgsli.com/MjQ2ODEx)

basicsr/__init__.py

@@ -0,0 +1,12 @@
+# https://github.com/xinntao/BasicSR
+# flake8: noqa
+from .archs import *
+from .data import *
+from .losses import *
+from .metrics import *
+from .models import *
+from .ops import *
+from .test import *
+from .train import *
+from .utils import *
+# from .version import __gitsha__, __version__

basicsr/archs/__init__.py

@@ -0,0 +1,24 @@
+import importlib
+from copy import deepcopy
+from os import path as osp
+
+from basicsr.utils import get_root_logger, scandir
+from basicsr.utils.registry import ARCH_REGISTRY
+
+__all__ = ['build_network']
+
+# automatically scan and import arch modules for registry
+# scan all the files under the 'archs' folder and collect files ending with '_arch.py'
+arch_folder = osp.dirname(osp.abspath(__file__))
+arch_filenames = [osp.splitext(osp.basename(v))[0] for v in scandir(arch_folder) if v.endswith('_arch.py')]
+# import all the arch modules
+_arch_modules = [importlib.import_module(f'basicsr.archs.{file_name}') for file_name in arch_filenames]
+
+
+def build_network(opt):
+    opt = deepcopy(opt)
+    network_type = opt.pop('type')
+    net = ARCH_REGISTRY.get(network_type)(**opt)
+    logger = get_root_logger()
+    logger.info(f'Network [{net.__class__.__name__}] is created.')
+    return net