Hugging Face's logo

waveydaveygravy
/
sd_vid

Model card Files
xet
Community
waveydaveygravy commited on
Commit
514eeec
·
1 Parent(s): 60bd9fd

Upload SD_vid_v3.ipynb

Browse files
Files changed (1) hide show
  1. SD_vid_v3.ipynb +1094 -0
SD_vid_v3.ipynb ADDED
@@ -0,0 +1,1094 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "metadata": {
6
+ "id": "QIuzds5LLPyC"
7
+ },
8
+ "source": [
9
+ "<a href=\"https://colab.research.google.com/github/mkshing/notebooks/blob/main/stable_video_diffusion_img2vid.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
10
+ "# Stable Video Diffusion (image-to-video) Demo\n",
11
+ "This notebook is the demo for the new image-to-video model, Stable Video Diffusion, from [Stability AI](https://stability.ai/) **on Colab free plan**.\n",
12
+ "\n",
13
+ "This was made by [mkshing](https://twitter.com/mk1stats).\n",
14
+ "\n",
15
+ "Visit the following links for the details of Stable Video Diffusion.\n",
16
+ "* Codebase: https://github.com/Stability-AI/generative-models\n",
17
+ "* HF: https://huggingface.co/stabilityai/stable-video-diffusion-img2vid\n",
18
+ " * LICENSE: [STABLE VIDEO DIFFUSION NON-COMMERCIAL COMMUNITY LICENSE AGREEMENT](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid/blob/main/LICENSE)\n",
19
+ "* Paper: https://stability.ai/research/stable-video-diffusion-scaling-latent-video-diffusion-models-to-large-datasets\n",
20
+ "\n",
21
+ "\n",
22
+ "\n",
23
+ "## Updates\n",
24
+ "### 2023.11.27\n",
25
+ "* Add the other hyper-parameters (`fps_id`, `motion_bucket_id`, `cond_aug`)\n"
26
+ ]
27
+ },
28
+ {
29
+ "cell_type": "code",
30
+ "source": [],
31
+ "metadata": {
32
+ "id": "_jO7XtApXOjS"
33
+ },
34
+ "execution_count": null,
35
+ "outputs": []
36
+ },
37
+ {
38
+ "cell_type": "code",
39
+ "execution_count": 1,
40
+ "metadata": {
41
+ "id": "aaimSFWfLPgb",
42
+ "outputId": "cec8dd8a-6ef5-4e4f-9ce6-2ad1c4990309",
43
+ "colab": {
44
+ "base_uri": "https://localhost:8080/",
45
+ "height": 1000
46
+ }
47
+ },
48
+ "outputs": [
49
+ {
50
+ "output_type": "stream",
51
+ "name": "stdout",
52
+ "text": [
53
+ "[Errno 2] No such file or directory: 'content'\n",
54
+ "/content\n",
55
+ "Tue Nov 28 09:53:58 2023 \n",
56
+ "+-----------------------------------------------------------------------------+\n",
57
+ "| NVIDIA-SMI 525.105.17 Driver Version: 525.105.17 CUDA Version: 12.0 |\n",
58
+ "|-------------------------------+----------------------+----------------------+\n",
59
+ "| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
60
+ "| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n",
61
+ "| | | MIG M. |\n",
62
+ "|===============================+======================+======================|\n",
63
+ "| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |\n",
64
+ "| N/A 52C P8 10W / 70W | 0MiB / 15360MiB | 0% Default |\n",
65
+ "| | | N/A |\n",
66
+ "+-------------------------------+----------------------+----------------------+\n",
67
+ " \n",
68
+ "+-----------------------------------------------------------------------------+\n",
69
+ "| Processes: |\n",
70
+ "| GPU GI CI PID Type Process name GPU Memory |\n",
71
+ "| ID ID Usage |\n",
72
+ "|=============================================================================|\n",
73
+ "| No running processes found |\n",
74
+ "+-----------------------------------------------------------------------------+\n",
75
+ "Cloning into 'generative-models'...\n",
76
+ "remote: Enumerating objects: 838, done.\u001b[K\n",
77
+ "remote: Counting objects: 100% (481/481), done.\u001b[K\n",
78
+ "remote: Compressing objects: 100% (218/218), done.\u001b[K\n",
79
+ "remote: Total 838 (delta 345), reused 309 (delta 260), pack-reused 357\u001b[K\n",
80
+ "Receiving objects: 100% (838/838), 40.60 MiB | 23.72 MiB/s, done.\n",
81
+ "Resolving deltas: 100% (421/421), done.\n",
82
+ "--2023-11-28 09:54:01-- https://gist.githubusercontent.com/mkshing/4ad40699756d996ba6b3f7934e6ca532/raw/3f0094272c7a2bd3eb5f1a0db91bed582c9e8f01/requirements.txt\n",
83
+ "Resolving gist.githubusercontent.com (gist.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
84
+ "Connecting to gist.githubusercontent.com (gist.githubusercontent.com)|185.199.108.133|:443... connected.\n",
85
+ "HTTP request sent, awaiting response... 200 OK\n",
86
+ "Length: 746 [text/plain]\n",
87
+ "Saving to: ‘requirements.txt’\n",
88
+ "\n",
89
+ "requirements.txt 100%[===================>] 746 --.-KB/s in 0s \n",
90
+ "\n",
91
+ "2023-11-28 09:54:01 (50.1 MB/s) - ‘requirements.txt’ saved [746/746]\n",
92
+ "\n",
93
+ "Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu118\n",
94
+ "Collecting clip@ git+https://github.com/openai/CLIP.git (from -r requirements.txt (line 4))\n",
95
+ " Cloning https://github.com/openai/CLIP.git to /tmp/pip-install-gw4n604c/clip_ce7128a2eaf04492b8fb7a6e281dd1fd\n",
96
+ " Running command git clone --filter=blob:none --quiet https://github.com/openai/CLIP.git /tmp/pip-install-gw4n604c/clip_ce7128a2eaf04492b8fb7a6e281dd1fd\n",
97
+ " Resolved https://github.com/openai/CLIP.git to commit a1d071733d7111c9c014f024669f959182114e33\n",
98
+ " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
99
+ "Collecting black==23.7.0 (from -r requirements.txt (line 2))\n",
100
+ " Downloading black-23.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)\n",
101
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m19.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
102
+ "\u001b[?25hCollecting chardet==5.1.0 (from -r requirements.txt (line 3))\n",
103
+ " Downloading chardet-5.1.0-py3-none-any.whl (199 kB)\n",
104
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m199.1/199.1 kB\u001b[0m \u001b[31m26.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
105
+ "\u001b[?25hCollecting einops>=0.6.1 (from -r requirements.txt (line 5))\n",
106
+ " Downloading einops-0.7.0-py3-none-any.whl (44 kB)\n",
107
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m44.6/44.6 kB\u001b[0m \u001b[31m5.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
108
+ "\u001b[?25hCollecting fairscale>=0.4.13 (from -r requirements.txt (line 6))\n",
109
+ " Downloading fairscale-0.4.13.tar.gz (266 kB)\n",
110
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m266.3/266.3 kB\u001b[0m \u001b[31m30.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
111
+ "\u001b[?25h Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
112
+ " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
113
+ " Installing backend dependencies ... \u001b[?25l\u001b[?25hdone\n",
114
+ " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
115
+ "Collecting fire>=0.5.0 (from -r requirements.txt (line 7))\n",
116
+ " Downloading fire-0.5.0.tar.gz (88 kB)\n",
117
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m88.3/88.3 kB\u001b[0m \u001b[31m11.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
118
+ "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
119
+ "Requirement already satisfied: fsspec>=2023.6.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 8)) (2023.6.0)\n",
120
+ "Collecting invisible-watermark>=0.2.0 (from -r requirements.txt (line 9))\n",
121
+ " Downloading invisible_watermark-0.2.0-py3-none-any.whl (1.6 MB)\n",
122
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m70.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
123
+ "\u001b[?25hCollecting kornia==0.6.9 (from -r requirements.txt (line 10))\n",
124
+ " Downloading kornia-0.6.9-py2.py3-none-any.whl (569 kB)\n",
125
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m569.1/569.1 kB\u001b[0m \u001b[31m48.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
126
+ "\u001b[?25hCollecting matplotlib>=3.7.2 (from -r requirements.txt (line 11))\n",
127
+ " Downloading matplotlib-3.8.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)\n",
128
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m11.6/11.6 MB\u001b[0m \u001b[31m84.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
129
+ "\u001b[?25hRequirement already satisfied: natsort>=8.4.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 12)) (8.4.0)\n",
130
+ "Collecting ninja>=1.11.1 (from -r requirements.txt (line 13))\n",
131
+ " Using cached ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (307 kB)\n",
132
+ "Collecting omegaconf>=2.3.0 (from -r requirements.txt (line 15))\n",
133
+ " Downloading omegaconf-2.3.0-py3-none-any.whl (79 kB)\n",
134
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m79.5/79.5 kB\u001b[0m \u001b[31m10.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
135
+ "\u001b[?25hCollecting open-clip-torch>=2.20.0 (from -r requirements.txt (line 16))\n",
136
+ " Downloading open_clip_torch-2.23.0-py3-none-any.whl (1.5 MB)\n",
137
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.5/1.5 MB\u001b[0m \u001b[31m86.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
138
+ "\u001b[?25hCollecting opencv-python==4.6.0.66 (from -r requirements.txt (line 17))\n",
139
+ " Downloading opencv_python-4.6.0.66-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (60.9 MB)\n",
140
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m60.9/60.9 MB\u001b[0m \u001b[31m10.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
141
+ "\u001b[?25hCollecting pandas>=2.0.3 (from -r requirements.txt (line 18))\n",
142
+ " Downloading pandas-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.3 MB)\n",
143
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m12.3/12.3 MB\u001b[0m \u001b[31m108.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
144
+ "\u001b[?25hCollecting pillow>=9.5.0 (from -r requirements.txt (line 19))\n",
145
+ " Downloading Pillow-10.1.0-cp310-cp310-manylinux_2_28_x86_64.whl (3.6 MB)\n",
146
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.6/3.6 MB\u001b[0m \u001b[31m80.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
147
+ "\u001b[?25hCollecting pudb>=2022.1.3 (from -r requirements.txt (line 20))\n",
148
+ " Downloading pudb-2023.1.tar.gz (224 kB)\n",
149
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m224.3/224.3 kB\u001b[0m \u001b[31m22.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
150
+ "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
151
+ "Collecting pytorch-lightning==2.0.1 (from -r requirements.txt (line 21))\n",
152
+ " Downloading pytorch_lightning-2.0.1-py3-none-any.whl (716 kB)\n",
153
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m716.4/716.4 kB\u001b[0m \u001b[31m68.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
154
+ "\u001b[?25hRequirement already satisfied: pyyaml>=6.0.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 22)) (6.0.1)\n",
155
+ "Requirement already satisfied: scipy>=1.10.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 23)) (1.11.3)\n",
156
+ "Collecting streamlit>=0.73.1 (from -r requirements.txt (line 24))\n",
157
+ " Downloading streamlit-1.28.2-py2.py3-none-any.whl (8.4 MB)\n",
158
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m8.4/8.4 MB\u001b[0m \u001b[31m123.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
159
+ "\u001b[?25hCollecting tensorboardx==2.6 (from -r requirements.txt (line 25))\n",
160
+ " Downloading tensorboardX-2.6-py2.py3-none-any.whl (114 kB)\n",
161
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m114.5/114.5 kB\u001b[0m \u001b[31m17.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
162
+ "\u001b[?25hCollecting timm>=0.9.2 (from -r requirements.txt (line 26))\n",
163
+ " Downloading timm-0.9.12-py3-none-any.whl (2.2 MB)\n",
164
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.2/2.2 MB\u001b[0m \u001b[31m108.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
165
+ "\u001b[?25hCollecting tokenizers==0.12.1 (from -r requirements.txt (line 27))\n",
166
+ " Downloading tokenizers-0.12.1-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.6 MB)\n",
167
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.6/6.6 MB\u001b[0m \u001b[31m120.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
168
+ "\u001b[?25hRequirement already satisfied: torch>=2.0.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 28)) (2.1.0+cu118)\n",
169
+ "Requirement already satisfied: torchaudio>=2.0.2 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 29)) (2.1.0+cu118)\n",
170
+ "Collecting torchdata==0.6.1 (from -r requirements.txt (line 30))\n",
171
+ " Downloading https://download.pytorch.org/whl/torchdata-0.6.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB)\n",
172
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.6/4.6 MB\u001b[0m \u001b[31m49.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
173
+ "\u001b[?25hCollecting torchmetrics>=1.0.1 (from -r requirements.txt (line 31))\n",
174
+ " Downloading torchmetrics-1.2.0-py3-none-any.whl (805 kB)\n",
175
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m805.2/805.2 kB\u001b[0m \u001b[31m73.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
176
+ "\u001b[?25hRequirement already satisfied: torchvision>=0.15.2 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 32)) (0.16.0+cu118)\n",
177
+ "Requirement already satisfied: tqdm>=4.65.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 33)) (4.66.1)\n",
178
+ "Collecting transformers==4.19.1 (from -r requirements.txt (line 34))\n",
179
+ " Downloading transformers-4.19.1-py3-none-any.whl (4.2 MB)\n",
180
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.2/4.2 MB\u001b[0m \u001b[31m117.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
181
+ "\u001b[?25hCollecting triton==2.0.0 (from -r requirements.txt (line 35))\n",
182
+ " Downloading https://download.pytorch.org/whl/triton-2.0.0-1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (63.3 MB)\n",
183
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m63.3/63.3 MB\u001b[0m \u001b[31m9.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
184
+ "\u001b[?25hCollecting urllib3<1.27,>=1.25.4 (from -r requirements.txt (line 36))\n",
185
+ " Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)\n",
186
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m143.8/143.8 kB\u001b[0m \u001b[31m20.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
187
+ "\u001b[?25hCollecting wandb>=0.15.6 (from -r requirements.txt (line 37))\n",
188
+ " Downloading wandb-0.16.0-py3-none-any.whl (2.1 MB)\n",
189
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.1/2.1 MB\u001b[0m \u001b[31m81.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
190
+ "\u001b[?25hCollecting webdataset>=0.2.33 (from -r requirements.txt (line 38))\n",
191
+ " Downloading webdataset-0.2.79-py3-none-any.whl (65 kB)\n",
192
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m65.4/65.4 kB\u001b[0m \u001b[31m7.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
193
+ "\u001b[?25hRequirement already satisfied: wheel>=0.41.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 39)) (0.41.3)\n",
194
+ "Collecting xformers>=0.0.20 (from -r requirements.txt (line 40))\n",
195
+ " Downloading https://download.pytorch.org/whl/cu118/xformers-0.0.22.post7%2Bcu118-cp310-cp310-manylinux2014_x86_64.whl (211.5 MB)\n",
196
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.5/211.5 MB\u001b[0m \u001b[31m5.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
197
+ "\u001b[?25hRequirement already satisfied: click>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from black==23.7.0->-r requirements.txt (line 2)) (8.1.7)\n",
198
+ "Collecting mypy-extensions>=0.4.3 (from black==23.7.0->-r requirements.txt (line 2))\n",
199
+ " Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
200
+ "Requirement already satisfied: packaging>=22.0 in /usr/local/lib/python3.10/dist-packages (from black==23.7.0->-r requirements.txt (line 2)) (23.2)\n",
201
+ "Collecting pathspec>=0.9.0 (from black==23.7.0->-r requirements.txt (line 2))\n",
202
+ " Downloading pathspec-0.11.2-py3-none-any.whl (29 kB)\n",
203
+ "Requirement already satisfied: platformdirs>=2 in /usr/local/lib/python3.10/dist-packages (from black==23.7.0->-r requirements.txt (line 2)) (4.0.0)\n",
204
+ "Requirement already satisfied: tomli>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from black==23.7.0->-r requirements.txt (line 2)) (2.0.1)\n",
205
+ "Requirement already satisfied: numpy>=1.21.2 in /usr/local/lib/python3.10/dist-packages (from opencv-python==4.6.0.66->-r requirements.txt (line 17)) (1.23.5)\n",
206
+ "Requirement already satisfied: typing-extensions>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from pytorch-lightning==2.0.1->-r requirements.txt (line 21)) (4.5.0)\n",
207
+ "Collecting lightning-utilities>=0.7.0 (from pytorch-lightning==2.0.1->-r requirements.txt (line 21))\n",
208
+ " Downloading lightning_utilities-0.10.0-py3-none-any.whl (24 kB)\n",
209
+ "Requirement already satisfied: protobuf<4,>=3.8.0 in /usr/local/lib/python3.10/dist-packages (from tensorboardx==2.6->-r requirements.txt (line 25)) (3.20.3)\n",
210
+ "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from torchdata==0.6.1->-r requirements.txt (line 30)) (2.31.0)\n",
211
+ "Collecting torch>=2.0.1 (from -r requirements.txt (line 28))\n",
212
+ " Downloading https://download.pytorch.org/whl/cu118/torch-2.0.1%2Bcu118-cp310-cp310-linux_x86_64.whl (2267.3 MB)\n",
213
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.3/2.3 GB\u001b[0m \u001b[31m549.6 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
214
+ "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=2.0.1->-r requirements.txt (line 28)) (3.13.1)\n",
215
+ "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=2.0.1->-r requirements.txt (line 28)) (1.12)\n",
216
+ "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=2.0.1->-r requirements.txt (line 28)) (3.2.1)\n",
217
+ "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=2.0.1->-r requirements.txt (line 28)) (3.1.2)\n",
218
+ "Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.19.1->-r requirements.txt (line 34)) (0.19.4)\n",
219
+ "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.19.1->-r requirements.txt (line 34)) (2023.6.3)\n",
220
+ "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->-r requirements.txt (line 35)) (3.27.7)\n",
221
+ "Collecting lit (from triton==2.0.0->-r requirements.txt (line 35))\n",
222
+ " Downloading lit-17.0.5.tar.gz (153 kB)\n",
223
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m153.0/153.0 kB\u001b[0m \u001b[31m20.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
224
+ "\u001b[?25h Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
225
+ " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
226
+ " Installing backend dependencies ... \u001b[?25l\u001b[?25hdone\n",
227
+ " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
228
+ "Collecting ftfy (from clip@ git+https://github.com/openai/CLIP.git->-r requirements.txt (line 4))\n",
229
+ " Downloading ftfy-6.1.3-py3-none-any.whl (53 kB)\n",
230
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.4/53.4 kB\u001b[0m \u001b[31m7.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
231
+ "\u001b[?25hRequirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from fire>=0.5.0->-r requirements.txt (line 7)) (1.16.0)\n",
232
+ "Requirement already satisfied: termcolor in /usr/local/lib/python3.10/dist-packages (from fire>=0.5.0->-r requirements.txt (line 7)) (2.3.0)\n",
233
+ "Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from invisible-watermark>=0.2.0->-r requirements.txt (line 9)) (1.4.1)\n",
234
+ "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.7.2->-r requirements.txt (line 11)) (1.2.0)\n",
235
+ "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.7.2->-r requirements.txt (line 11)) (0.12.1)\n",
236
+ "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.7.2->-r requirements.txt (line 11)) (4.44.3)\n",
237
+ "Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.7.2->-r requirements.txt (line 11)) (1.4.5)\n",
238
+ "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.7.2->-r requirements.txt (line 11)) (3.1.1)\n",
239
+ "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=3.7.2->-r requirements.txt (line 11)) (2.8.2)\n",
240
+ "Collecting antlr4-python3-runtime==4.9.* (from omegaconf>=2.3.0->-r requirements.txt (line 15))\n",
241
+ " Downloading antlr4-python3-runtime-4.9.3.tar.gz (117 kB)\n",
242
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m117.0/117.0 kB\u001b[0m \u001b[31m17.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
243
+ "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
244
+ "Collecting sentencepiece (from open-clip-torch>=2.20.0->-r requirements.txt (line 16))\n",
245
+ " Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
246
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m58.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
247
+ "\u001b[?25hRequirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=2.0.3->-r requirements.txt (line 18)) (2023.3.post1)\n",
248
+ "Collecting tzdata>=2022.1 (from pandas>=2.0.3->-r requirements.txt (line 18))\n",
249
+ " Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB)\n",
250
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m341.8/341.8 kB\u001b[0m \u001b[31m35.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
251
+ "\u001b[?25hCollecting urwid>=1.1.1 (from pudb>=2022.1.3->-r requirements.txt (line 20))\n",
252
+ " Downloading urwid-2.2.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (274 kB)\n",
253
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m274.4/274.4 kB\u001b[0m \u001b[31m28.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
254
+ "\u001b[?25hRequirement already satisfied: pygments>=2.7.4 in /usr/local/lib/python3.10/dist-packages (from pudb>=2022.1.3->-r requirements.txt (line 20)) (2.16.1)\n",
255
+ "Collecting jedi<1,>=0.18 (from pudb>=2022.1.3->-r requirements.txt (line 20))\n",
256
+ " Downloading jedi-0.19.1-py2.py3-none-any.whl (1.6 MB)\n",
257
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m54.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
258
+ "\u001b[?25hCollecting urwid_readline (from pudb>=2022.1.3->-r requirements.txt (line 20))\n",
259
+ " Downloading urwid_readline-0.13.tar.gz (7.9 kB)\n",
260
+ " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
261
+ " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
262
+ " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
263
+ "Requirement already satisfied: altair<6,>=4.0 in /usr/local/lib/python3.10/dist-packages (from streamlit>=0.73.1->-r requirements.txt (line 24)) (4.2.2)\n",
264
+ "Requirement already satisfied: blinker<2,>=1.0.0 in /usr/lib/python3/dist-packages (from streamlit>=0.73.1->-r requirements.txt (line 24)) (1.4)\n",
265
+ "Requirement already satisfied: cachetools<6,>=4.0 in /usr/local/lib/python3.10/dist-packages (from streamlit>=0.73.1->-r requirements.txt (line 24)) (5.3.2)\n",
266
+ "Requirement already satisfied: importlib-metadata<7,>=1.4 in /usr/local/lib/python3.10/dist-packages (from streamlit>=0.73.1->-r requirements.txt (line 24)) (6.8.0)\n",
267
+ "Requirement already satisfied: pyarrow>=6.0 in /usr/local/lib/python3.10/dist-packages (from streamlit>=0.73.1->-r requirements.txt (line 24)) (9.0.0)\n",
268
+ "Requirement already satisfied: rich<14,>=10.14.0 in /usr/local/lib/python3.10/dist-packages (from streamlit>=0.73.1->-r requirements.txt (line 24)) (13.7.0)\n",
269
+ "Requirement already satisfied: tenacity<9,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from streamlit>=0.73.1->-r requirements.txt (line 24)) (8.2.3)\n",
270
+ "Requirement already satisfied: toml<2,>=0.10.1 in /usr/local/lib/python3.10/dist-packages (from streamlit>=0.73.1->-r requirements.txt (line 24)) (0.10.2)\n",
271
+ "Requirement already satisfied: tzlocal<6,>=1.1 in /usr/local/lib/python3.10/dist-packages (from streamlit>=0.73.1->-r requirements.txt (line 24)) (5.2)\n",
272
+ "Collecting validators<1,>=0.2 (from streamlit>=0.73.1->-r requirements.txt (line 24))\n",
273
+ " Downloading validators-0.22.0-py3-none-any.whl (26 kB)\n",
274
+ "Collecting gitpython!=3.1.19,<4,>=3.0.7 (from streamlit>=0.73.1->-r requirements.txt (line 24))\n",
275
+ " Downloading GitPython-3.1.40-py3-none-any.whl (190 kB)\n",
276
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.6/190.6 kB\u001b[0m \u001b[31m26.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
277
+ "\u001b[?25hCollecting pydeck<1,>=0.8.0b4 (from streamlit>=0.73.1->-r requirements.txt (line 24))\n",
278
+ " Downloading pydeck-0.8.1b0-py2.py3-none-any.whl (4.8 MB)\n",
279
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.8/4.8 MB\u001b[0m \u001b[31m121.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
280
+ "\u001b[?25hRequirement already satisfied: tornado<7,>=6.0.3 in /usr/local/lib/python3.10/dist-packages (from streamlit>=0.73.1->-r requirements.txt (line 24)) (6.3.2)\n",
281
+ "Collecting watchdog>=2.1.5 (from streamlit>=0.73.1->-r requirements.txt (line 24))\n",
282
+ " Downloading watchdog-3.0.0-py3-none-manylinux2014_x86_64.whl (82 kB)\n",
283
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m82.1/82.1 kB\u001b[0m \u001b[31m12.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
284
+ "\u001b[?25hRequirement already satisfied: safetensors in /usr/local/lib/python3.10/dist-packages (from timm>=0.9.2->-r requirements.txt (line 26)) (0.4.0)\n",
285
+ "INFO: pip is looking at multiple versions of torchaudio to determine which version is compatible with other requirements. This could take a while.\n",
286
+ "Collecting torchaudio>=2.0.2 (from -r requirements.txt (line 29))\n",
287
+ " Downloading https://download.pytorch.org/whl/cu118/torchaudio-2.1.1%2Bcu118-cp310-cp310-linux_x86_64.whl (3.2 MB)\n",
288
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.2/3.2 MB\u001b[0m \u001b[31m105.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
289
+ "\u001b[?25h Downloading torchaudio-2.1.1-cp310-cp310-manylinux1_x86_64.whl (3.3 MB)\n",
290
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.3/3.3 MB\u001b[0m \u001b[31m107.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
291
+ "\u001b[?25h Downloading torchaudio-2.1.0-cp310-cp310-manylinux1_x86_64.whl (3.3 MB)\n",
292
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.3/3.3 MB\u001b[0m \u001b[31m112.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
293
+ "\u001b[?25h Downloading https://download.pytorch.org/whl/cu118/torchaudio-2.0.2%2Bcu118-cp310-cp310-linux_x86_64.whl (4.4 MB)\n",
294
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.4/4.4 MB\u001b[0m \u001b[31m111.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
295
+ "\u001b[?25hINFO: pip is looking at multiple versions of torchvision to determine which version is compatible with other requirements. This could take a while.\n",
296
+ "Collecting torchvision>=0.15.2 (from -r requirements.txt (line 32))\n",
297
+ " Downloading https://download.pytorch.org/whl/cu118/torchvision-0.16.1%2Bcu118-cp310-cp310-linux_x86_64.whl (6.1 MB)\n",
298
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.1/6.1 MB\u001b[0m \u001b[31m120.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
299
+ "\u001b[?25h Downloading torchvision-0.16.1-cp310-cp310-manylinux1_x86_64.whl (6.8 MB)\n",
300
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.8/6.8 MB\u001b[0m \u001b[31m113.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
301
+ "\u001b[?25h Downloading torchvision-0.16.0-cp310-cp310-manylinux1_x86_64.whl (6.9 MB)\n",
302
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.9/6.9 MB\u001b[0m \u001b[31m120.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
303
+ "\u001b[?25h Downloading https://download.pytorch.org/whl/cu118/torchvision-0.15.2%2Bcu118-cp310-cp310-linux_x86_64.whl (6.1 MB)\n",
304
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.1/6.1 MB\u001b[0m \u001b[31m121.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
305
+ "\u001b[?25hRequirement already satisfied: psutil>=5.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb>=0.15.6->-r requirements.txt (line 37)) (5.9.5)\n",
306
+ "Collecting sentry-sdk>=1.0.0 (from wandb>=0.15.6->-r requirements.txt (line 37))\n",
307
+ " Downloading sentry_sdk-1.37.1-py2.py3-none-any.whl (251 kB)\n",
308
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m251.7/251.7 kB\u001b[0m \u001b[31m34.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
309
+ "\u001b[?25hCollecting docker-pycreds>=0.4.0 (from wandb>=0.15.6->-r requirements.txt (line 37))\n",
310
+ " Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)\n",
311
+ "Collecting setproctitle (from wandb>=0.15.6->-r requirements.txt (line 37))\n",
312
+ " Downloading setproctitle-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30 kB)\n",
313
+ "Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from wandb>=0.15.6->-r requirements.txt (line 37)) (67.7.2)\n",
314
+ "Requirement already satisfied: appdirs>=1.4.3 in /usr/local/lib/python3.10/dist-packages (from wandb>=0.15.6->-r requirements.txt (line 37)) (1.4.4)\n",
315
+ "Collecting braceexpand (from webdataset>=0.2.33->-r requirements.txt (line 38))\n",
316
+ " Downloading braceexpand-0.1.7-py2.py3-none-any.whl (5.9 kB)\n",
317
+ "INFO: pip is looking at multiple versions of xformers to determine which version is compatible with other requirements. This could take a while.\n",
318
+ "Collecting xformers>=0.0.20 (from -r requirements.txt (line 40))\n",
319
+ " Downloading xformers-0.0.22.post7-cp310-cp310-manylinux2014_x86_64.whl (211.8 MB)\n",
320
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.8/211.8 MB\u001b[0m \u001b[31m5.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
321
+ "\u001b[?25h Downloading https://download.pytorch.org/whl/cu118/xformers-0.0.22.post4%2Bcu118-cp310-cp310-manylinux2014_x86_64.whl (211.4 MB)\n",
322
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.4/211.4 MB\u001b[0m \u001b[31m6.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
323
+ "\u001b[?25h Downloading xformers-0.0.22.post4-cp310-cp310-manylinux2014_x86_64.whl (211.8 MB)\n",
324
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.8/211.8 MB\u001b[0m \u001b[31m5.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
325
+ "\u001b[?25h Downloading https://download.pytorch.org/whl/cu118/xformers-0.0.22.post3%2Bcu118-cp310-cp310-manylinux2014_x86_64.whl (211.5 MB)\n",
326
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.5/211.5 MB\u001b[0m \u001b[31m6.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
327
+ "\u001b[?25h Downloading xformers-0.0.22-cp310-cp310-manylinux2014_x86_64.whl (211.6 MB)\n",
328
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.6/211.6 MB\u001b[0m \u001b[31m6.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
329
+ "\u001b[?25hRequirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6,>=4.0->streamlit>=0.73.1->-r requirements.txt (line 24)) (0.4)\n",
330
+ "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6,>=4.0->streamlit>=0.73.1->-r requirements.txt (line 24)) (4.19.2)\n",
331
+ "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6,>=4.0->streamlit>=0.73.1->-r requirements.txt (line 24)) (0.12.0)\n",
332
+ "Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /usr/local/lib/python3.10/dist-packages (from fsspec>=2023.6.0->-r requirements.txt (line 8)) (3.8.6)\n",
333
+ "Collecting gitdb<5,>=4.0.1 (from gitpython!=3.1.19,<4,>=3.0.7->streamlit>=0.73.1->-r requirements.txt (line 24))\n",
334
+ " Downloading gitdb-4.0.11-py3-none-any.whl (62 kB)\n",
335
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.7/62.7 kB\u001b[0m \u001b[31m8.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
336
+ "\u001b[?25hRequirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata<7,>=1.4->streamlit>=0.73.1->-r requirements.txt (line 24)) (3.17.0)\n",
337
+ "Requirement already satisfied: parso<0.9.0,>=0.8.3 in /usr/local/lib/python3.10/dist-packages (from jedi<1,>=0.18->pudb>=2022.1.3->-r requirements.txt (line 20)) (0.8.3)\n",
338
+ "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=2.0.1->-r requirements.txt (line 28)) (2.1.3)\n",
339
+ "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->torchdata==0.6.1->-r requirements.txt (line 30)) (3.3.2)\n",
340
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->torchdata==0.6.1->-r requirements.txt (line 30)) (3.4)\n",
341
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->torchdata==0.6.1->-r requirements.txt (line 30)) (2023.7.22)\n",
342
+ "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich<14,>=10.14.0->streamlit>=0.73.1->-r requirements.txt (line 24)) (3.0.0)\n",
343
+ "Collecting wcwidth<0.3.0,>=0.2.12 (from ftfy->clip@ git+https://github.com/openai/CLIP.git->-r requirements.txt (line 4))\n",
344
+ " Downloading wcwidth-0.2.12-py2.py3-none-any.whl (34 kB)\n",
345
+ "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=2.0.1->-r requirements.txt (line 28)) (1.3.0)\n",
346
+ "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec>=2023.6.0->-r requirements.txt (line 8)) (23.1.0)\n",
347
+ "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec>=2023.6.0->-r requirements.txt (line 8)) (6.0.4)\n",
348
+ "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec>=2023.6.0->-r requirements.txt (line 8)) (4.0.3)\n",
349
+ "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec>=2023.6.0->-r requirements.txt (line 8)) (1.9.2)\n",
350
+ "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec>=2023.6.0->-r requirements.txt (line 8)) (1.4.0)\n",
351
+ "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec>=2023.6.0->-r requirements.txt (line 8)) (1.3.1)\n",
352
+ "Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->gitpython!=3.1.19,<4,>=3.0.7->streamlit>=0.73.1->-r requirements.txt (line 24))\n",
353
+ " Downloading smmap-5.0.1-py3-none-any.whl (24 kB)\n",
354
+ "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit>=0.73.1->-r requirements.txt (line 24)) (2023.11.1)\n",
355
+ "Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit>=0.73.1->-r requirements.txt (line 24)) (0.31.0)\n",
356
+ "Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit>=0.73.1->-r requirements.txt (line 24)) (0.13.0)\n",
357
+ "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich<14,>=10.14.0->streamlit>=0.73.1->-r requirements.txt (line 24)) (0.1.2)\n",
358
+ "Building wheels for collected packages: clip, fairscale, fire, antlr4-python3-runtime, pudb, lit, urwid_readline\n",
359
+ " Building wheel for clip (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
360
+ " Created wheel for clip: filename=clip-1.0-py3-none-any.whl size=1369500 sha256=8a926b6bfd2535ed91a092ed200281fe8791862f976e98e41513ad30b94341ab\n",
361
+ " Stored in directory: /tmp/pip-ephem-wheel-cache-jqnd6upq/wheels/da/2b/4c/d6691fa9597aac8bb85d2ac13b112deb897d5b50f5ad9a37e4\n",
362
+ " Building wheel for fairscale (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
363
+ " Created wheel for fairscale: filename=fairscale-0.4.13-py3-none-any.whl size=332104 sha256=4c519aa716e61c0c20d399b56e7594616f17be56a336b7f13bb3131e4201a796\n",
364
+ " Stored in directory: /root/.cache/pip/wheels/78/a4/c0/fb0a7ef03cff161611c3fa40c6cf898f76e58ec421b88e8cb3\n",
365
+ " Building wheel for fire (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
366
+ " Created wheel for fire: filename=fire-0.5.0-py2.py3-none-any.whl size=116933 sha256=c28df1c1febb699eeda0f8ab2775339d1979ef38a9eb545da022ffd8a89ccb6b\n",
367
+ " Stored in directory: /root/.cache/pip/wheels/90/d4/f7/9404e5db0116bd4d43e5666eaa3e70ab53723e1e3ea40c9a95\n",
368
+ " Building wheel for antlr4-python3-runtime (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
369
+ " Created wheel for antlr4-python3-runtime: filename=antlr4_python3_runtime-4.9.3-py3-none-any.whl size=144555 sha256=6370b74f4b06d69fbd8723ce802c56ebf7bd170cd62dd31efb5ad69f9c1c356c\n",
370
+ " Stored in directory: /root/.cache/pip/wheels/12/93/dd/1f6a127edc45659556564c5730f6d4e300888f4bca2d4c5a88\n",
371
+ " Building wheel for pudb (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
372
+ " Created wheel for pudb: filename=pudb-2023.1-py3-none-any.whl size=86116 sha256=84f77a278fb0168f6d632d38b9c8f11590b51dae60033f6914b56420580fdb4d\n",
373
+ " Stored in directory: /root/.cache/pip/wheels/50/03/e9/97cb1ce91fb1601f85470b71f11fcd3c6617e81735ccd4460c\n",
374
+ " Building wheel for lit (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
375
+ " Created wheel for lit: filename=lit-17.0.5-py3-none-any.whl size=93256 sha256=fefe9683dd896b8b1b4452b81640eee67238a2b03ebd25454ac0c58c618f5a2a\n",
376
+ " Stored in directory: /root/.cache/pip/wheels/1c/87/8e/5a42c0d4be23362b68bbff33b17f3c35a3df44f1cd2f5a24b4\n",
377
+ " Building wheel for urwid_readline (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
378
+ " Created wheel for urwid_readline: filename=urwid_readline-0.13-py3-none-any.whl size=7549 sha256=9b1b69c2e94743da9f09f0e8dbda2ff72e2ef94e584a99b21a9c6b2fa7c2a73a\n",
379
+ " Stored in directory: /root/.cache/pip/wheels/58/1d/d8/20c6d76afd5bd205f5f95f19640df9a4e88fc6f1a4c25bb693\n",
380
+ "Successfully built clip fairscale fire antlr4-python3-runtime pudb lit urwid_readline\n",
381
+ "Installing collected packages: wcwidth, tokenizers, sentencepiece, ninja, lit, braceexpand, antlr4-python3-runtime, webdataset, watchdog, validators, urwid, urllib3, tzdata, tensorboardx, smmap, setproctitle, pillow, pathspec, opencv-python, omegaconf, mypy-extensions, lightning-utilities, jedi, ftfy, fire, einops, docker-pycreds, chardet, urwid_readline, sentry-sdk, pydeck, pandas, matplotlib, gitdb, black, pudb, gitpython, wandb, transformers, streamlit, triton, torch, torchvision, torchmetrics, timm, xformers, torchdata, torchaudio, pytorch-lightning, open-clip-torch, kornia, invisible-watermark, fairscale, clip\n",
382
+ " Attempting uninstall: wcwidth\n",
383
+ " Found existing installation: wcwidth 0.2.10\n",
384
+ " Uninstalling wcwidth-0.2.10:\n",
385
+ " Successfully uninstalled wcwidth-0.2.10\n",
386
+ " Attempting uninstall: tokenizers\n",
387
+ " Found existing installation: tokenizers 0.15.0\n",
388
+ " Uninstalling tokenizers-0.15.0:\n",
389
+ " Successfully uninstalled tokenizers-0.15.0\n",
390
+ " Attempting uninstall: urllib3\n",
391
+ " Found existing installation: urllib3 2.0.7\n",
392
+ " Uninstalling urllib3-2.0.7:\n",
393
+ " Successfully uninstalled urllib3-2.0.7\n",
394
+ " Attempting uninstall: pillow\n",
395
+ " Found existing installation: Pillow 9.4.0\n",
396
+ " Uninstalling Pillow-9.4.0:\n",
397
+ " Successfully uninstalled Pillow-9.4.0\n",
398
+ " Attempting uninstall: opencv-python\n",
399
+ " Found existing installation: opencv-python 4.8.0.76\n",
400
+ " Uninstalling opencv-python-4.8.0.76:\n",
401
+ " Successfully uninstalled opencv-python-4.8.0.76\n",
402
+ " Attempting uninstall: chardet\n",
403
+ " Found existing installation: chardet 5.2.0\n",
404
+ " Uninstalling chardet-5.2.0:\n",
405
+ " Successfully uninstalled chardet-5.2.0\n",
406
+ " Attempting uninstall: pandas\n",
407
+ " Found existing installation: pandas 1.5.3\n",
408
+ " Uninstalling pandas-1.5.3:\n",
409
+ " Successfully uninstalled pandas-1.5.3\n",
410
+ " Attempting uninstall: matplotlib\n",
411
+ " Found existing installation: matplotlib 3.7.1\n",
412
+ " Uninstalling matplotlib-3.7.1:\n",
413
+ " Successfully uninstalled matplotlib-3.7.1\n",
414
+ " Attempting uninstall: transformers\n",
415
+ " Found existing installation: transformers 4.35.2\n",
416
+ " Uninstalling transformers-4.35.2:\n",
417
+ " Successfully uninstalled transformers-4.35.2\n",
418
+ " Attempting uninstall: triton\n",
419
+ " Found existing installation: triton 2.1.0\n",
420
+ " Uninstalling triton-2.1.0:\n",
421
+ " Successfully uninstalled triton-2.1.0\n",
422
+ " Attempting uninstall: torch\n",
423
+ " Found existing installation: torch 2.1.0+cu118\n",
424
+ " Uninstalling torch-2.1.0+cu118:\n",
425
+ " Successfully uninstalled torch-2.1.0+cu118\n",
426
+ " Attempting uninstall: torchvision\n",
427
+ " Found existing installation: torchvision 0.16.0+cu118\n",
428
+ " Uninstalling torchvision-0.16.0+cu118:\n",
429
+ " Successfully uninstalled torchvision-0.16.0+cu118\n",
430
+ " Attempting uninstall: torchdata\n",
431
+ " Found existing installation: torchdata 0.7.0\n",
432
+ " Uninstalling torchdata-0.7.0:\n",
433
+ " Successfully uninstalled torchdata-0.7.0\n",
434
+ " Attempting uninstall: torchaudio\n",
435
+ " Found existing installation: torchaudio 2.1.0+cu118\n",
436
+ " Uninstalling torchaudio-2.1.0+cu118:\n",
437
+ " Successfully uninstalled torchaudio-2.1.0+cu118\n",
438
+ "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
439
+ "lida 0.0.10 requires fastapi, which is not installed.\n",
440
+ "lida 0.0.10 requires kaleido, which is not installed.\n",
441
+ "lida 0.0.10 requires python-multipart, which is not installed.\n",
442
+ "lida 0.0.10 requires uvicorn, which is not installed.\n",
443
+ "google-colab 1.0.0 requires pandas==1.5.3, but you have pandas 2.1.3 which is incompatible.\n",
444
+ "imageio 2.31.6 requires pillow<10.1.0,>=8.3.2, but you have pillow 10.1.0 which is incompatible.\n",
445
+ "torchtext 0.16.0 requires torch==2.1.0, but you have torch 2.0.1+cu118 which is incompatible.\n",
446
+ "torchtext 0.16.0 requires torchdata==0.7.0, but you have torchdata 0.6.1 which is incompatible.\u001b[0m\u001b[31m\n",
447
+ "\u001b[0mSuccessfully installed antlr4-python3-runtime-4.9.3 black-23.7.0 braceexpand-0.1.7 chardet-5.1.0 clip-1.0 docker-pycreds-0.4.0 einops-0.7.0 fairscale-0.4.13 fire-0.5.0 ftfy-6.1.3 gitdb-4.0.11 gitpython-3.1.40 invisible-watermark-0.2.0 jedi-0.19.1 kornia-0.6.9 lightning-utilities-0.10.0 lit-17.0.5 matplotlib-3.8.2 mypy-extensions-1.0.0 ninja-1.11.1.1 omegaconf-2.3.0 open-clip-torch-2.23.0 opencv-python-4.6.0.66 pandas-2.1.3 pathspec-0.11.2 pillow-10.1.0 pudb-2023.1 pydeck-0.8.1b0 pytorch-lightning-2.0.1 sentencepiece-0.1.99 sentry-sdk-1.37.1 setproctitle-1.3.3 smmap-5.0.1 streamlit-1.28.2 tensorboardx-2.6 timm-0.9.12 tokenizers-0.12.1 torch-2.0.1+cu118 torchaudio-2.0.2+cu118 torchdata-0.6.1 torchmetrics-1.2.0 torchvision-0.15.2+cu118 transformers-4.19.1 triton-2.0.0 tzdata-2023.3 urllib3-1.26.18 urwid-2.2.3 urwid_readline-0.13 validators-0.22.0 wandb-0.16.0 watchdog-3.0.0 wcwidth-0.2.12 webdataset-0.2.79 xformers-0.0.22\n"
448
+ ]
449
+ },
450
+ {
451
+ "output_type": "display_data",
452
+ "data": {
453
+ "application/vnd.colab-display-data+json": {
454
+ "pip_warning": {
455
+ "packages": [
456
+ "PIL",
457
+ "matplotlib",
458
+ "mpl_toolkits",
459
+ "pydevd_plugins",
460
+ "wcwidth"
461
+ ]
462
+ }
463
+ }
464
+ },
465
+ "metadata": {}
466
+ },
467
+ {
468
+ "output_type": "stream",
469
+ "name": "stdout",
470
+ "text": [
471
+ "Obtaining file:///content/generative-models\n",
472
+ " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
473
+ " Checking if build backend supports build_editable ... \u001b[?25l\u001b[?25hdone\n",
474
+ " Getting requirements to build editable ... \u001b[?25l\u001b[?25hdone\n",
475
+ " Preparing editable metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
476
+ "Building wheels for collected packages: sgm\n",
477
+ " Building editable for sgm (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
478
+ " Created wheel for sgm: filename=sgm-0.1.0-py3-none-any.whl size=26737 sha256=afa2dcf74340e017d291435d122d245349fd3d0ccd82e8eab2e319f229829159\n",
479
+ " Stored in directory: /tmp/pip-ephem-wheel-cache-tdbtt00f/wheels/12/9b/27/03142f4dee9fa0a99f6c146eae81eb66e17b781145ecb05fa5\n",
480
+ "Successfully built sgm\n",
481
+ "Installing collected packages: sgm\n",
482
+ "Successfully installed sgm-0.1.0\n",
483
+ "Obtaining sdata from git+https://github.com/Stability-AI/datapipelines.git@main#egg=sdata\n",
484
+ " Cloning https://github.com/Stability-AI/datapipelines.git (to revision main) to ./src/sdata\n",
485
+ " Running command git clone --filter=blob:none --quiet https://github.com/Stability-AI/datapipelines.git /content/src/sdata\n",
486
+ " Resolved https://github.com/Stability-AI/datapipelines.git to commit 8bce77d147033b3a5285b6d45ee85f33866964fc\n",
487
+ " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
488
+ "Installing collected packages: sdata\n",
489
+ " Running setup.py develop for sdata\n",
490
+ "Successfully installed sdata-0.0.1\n",
491
+ "Collecting gradio\n",
492
+ " Downloading gradio-4.7.1-py3-none-any.whl (16.5 MB)\n",
493
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m16.5/16.5 MB\u001b[0m \u001b[31m86.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
494
+ "\u001b[?25hCollecting aiofiles<24.0,>=22.0 (from gradio)\n",
495
+ " Downloading aiofiles-23.2.1-py3-none-any.whl (15 kB)\n",
496
+ "Requirement already satisfied: altair<6.0,>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (4.2.2)\n",
497
+ "Collecting fastapi (from gradio)\n",
498
+ " Downloading fastapi-0.104.1-py3-none-any.whl (92 kB)\n",
499
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.9/92.9 kB\u001b[0m \u001b[31m14.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
500
+ "\u001b[?25hCollecting ffmpy (from gradio)\n",
501
+ " Downloading ffmpy-0.3.1.tar.gz (5.5 kB)\n",
502
+ " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
503
+ "Collecting gradio-client==0.7.0 (from gradio)\n",
504
+ " Downloading gradio_client-0.7.0-py3-none-any.whl (302 kB)\n",
505
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m302.7/302.7 kB\u001b[0m \u001b[31m35.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
506
+ "\u001b[?25hCollecting httpx (from gradio)\n",
507
+ " Downloading httpx-0.25.2-py3-none-any.whl (74 kB)\n",
508
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.0/75.0 kB\u001b[0m \u001b[31m11.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
509
+ "\u001b[?25hRequirement already satisfied: huggingface-hub>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (0.19.4)\n",
510
+ "Requirement already satisfied: importlib-resources<7.0,>=1.3 in /usr/local/lib/python3.10/dist-packages (from gradio) (6.1.1)\n",
511
+ "Requirement already satisfied: jinja2<4.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (3.1.2)\n",
512
+ "Requirement already satisfied: markupsafe~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (2.1.3)\n",
513
+ "Requirement already satisfied: matplotlib~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (3.8.2)\n",
514
+ "Requirement already satisfied: numpy~=1.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (1.23.5)\n",
515
+ "Collecting orjson~=3.0 (from gradio)\n",
516
+ " Downloading orjson-3.9.10-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)\n",
517
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.7/138.7 kB\u001b[0m \u001b[31m19.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
518
+ "\u001b[?25hRequirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from gradio) (23.2)\n",
519
+ "Requirement already satisfied: pandas<3.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (2.1.3)\n",
520
+ "Requirement already satisfied: pillow<11.0,>=8.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (10.1.0)\n",
521
+ "Collecting pydantic>=2.0 (from gradio)\n",
522
+ " Downloading pydantic-2.5.2-py3-none-any.whl (381 kB)\n",
523
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m381.9/381.9 kB\u001b[0m \u001b[31m40.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
524
+ "\u001b[?25hCollecting pydub (from gradio)\n",
525
+ " Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)\n",
526
+ "Collecting python-multipart (from gradio)\n",
527
+ " Downloading python_multipart-0.0.6-py3-none-any.whl (45 kB)\n",
528
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.7/45.7 kB\u001b[0m \u001b[31m6.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
529
+ "\u001b[?25hRequirement already satisfied: pyyaml<7.0,>=5.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (6.0.1)\n",
530
+ "Requirement already satisfied: requests~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (2.31.0)\n",
531
+ "Collecting semantic-version~=2.0 (from gradio)\n",
532
+ " Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)\n",
533
+ "Collecting tomlkit==0.12.0 (from gradio)\n",
534
+ " Downloading tomlkit-0.12.0-py3-none-any.whl (37 kB)\n",
535
+ "Requirement already satisfied: typer[all]<1.0,>=0.9 in /usr/local/lib/python3.10/dist-packages (from gradio) (0.9.0)\n",
536
+ "Requirement already satisfied: typing-extensions~=4.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (4.5.0)\n",
537
+ "Collecting uvicorn>=0.14.0 (from gradio)\n",
538
+ " Downloading uvicorn-0.24.0.post1-py3-none-any.whl (59 kB)\n",
539
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m59.7/59.7 kB\u001b[0m \u001b[31m8.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
540
+ "\u001b[?25hRequirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio-client==0.7.0->gradio) (2023.6.0)\n",
541
+ "Collecting websockets<12.0,>=10.0 (from gradio-client==0.7.0->gradio)\n",
542
+ " Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)\n",
543
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.9/129.9 kB\u001b[0m \u001b[31m17.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
544
+ "\u001b[?25hRequirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio) (0.4)\n",
545
+ "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio) (4.19.2)\n",
546
+ "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio) (0.12.0)\n",
547
+ "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.14.0->gradio) (3.13.1)\n",
548
+ "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.14.0->gradio) (4.66.1)\n",
549
+ "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (1.2.0)\n",
550
+ "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (0.12.1)\n",
551
+ "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (4.44.3)\n",
552
+ "Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (1.4.5)\n",
553
+ "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (3.1.1)\n",
554
+ "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (2.8.2)\n",
555
+ "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas<3.0,>=1.0->gradio) (2023.3.post1)\n",
556
+ "Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas<3.0,>=1.0->gradio) (2023.3)\n",
557
+ "Collecting annotated-types>=0.4.0 (from pydantic>=2.0->gradio)\n",
558
+ " Downloading annotated_types-0.6.0-py3-none-any.whl (12 kB)\n",
559
+ "Collecting pydantic-core==2.14.5 (from pydantic>=2.0->gradio)\n",
560
+ " Downloading pydantic_core-2.14.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)\n",
561
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.1/2.1 MB\u001b[0m \u001b[31m96.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
562
+ "\u001b[?25hCollecting typing-extensions~=4.0 (from gradio)\n",
563
+ " Downloading typing_extensions-4.8.0-py3-none-any.whl (31 kB)\n",
564
+ "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio) (3.3.2)\n",
565
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio) (3.4)\n",
566
+ "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio) (1.26.18)\n",
567
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio) (2023.7.22)\n",
568
+ "Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.10/dist-packages (from typer[all]<1.0,>=0.9->gradio) (8.1.7)\n",
569
+ "Collecting colorama<0.5.0,>=0.4.3 (from typer[all]<1.0,>=0.9->gradio)\n",
570
+ " Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)\n",
571
+ "Collecting shellingham<2.0.0,>=1.3.0 (from typer[all]<1.0,>=0.9->gradio)\n",
572
+ " Downloading shellingham-1.5.4-py2.py3-none-any.whl (9.8 kB)\n",
573
+ "Requirement already satisfied: rich<14.0.0,>=10.11.0 in /usr/local/lib/python3.10/dist-packages (from typer[all]<1.0,>=0.9->gradio) (13.7.0)\n",
574
+ "Collecting h11>=0.8 (from uvicorn>=0.14.0->gradio)\n",
575
+ " Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
576
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m9.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
577
+ "\u001b[?25hRequirement already satisfied: anyio<4.0.0,>=3.7.1 in /usr/local/lib/python3.10/dist-packages (from fastapi->gradio) (3.7.1)\n",
578
+ "Collecting starlette<0.28.0,>=0.27.0 (from fastapi->gradio)\n",
579
+ " Downloading starlette-0.27.0-py3-none-any.whl (66 kB)\n",
580
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.0/67.0 kB\u001b[0m \u001b[31m8.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
581
+ "\u001b[?25hCollecting httpcore==1.* (from httpx->gradio)\n",
582
+ " Downloading httpcore-1.0.2-py3-none-any.whl (76 kB)\n",
583
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.9/76.9 kB\u001b[0m \u001b[31m9.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
584
+ "\u001b[?25hRequirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio) (1.3.0)\n",
585
+ "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<4.0.0,>=3.7.1->fastapi->gradio) (1.1.3)\n",
586
+ "Requirement already satisfied: attrs>=22.2.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio) (23.1.0)\n",
587
+ "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio) (2023.11.1)\n",
588
+ "Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio) (0.31.0)\n",
589
+ "Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio) (0.13.0)\n",
590
+ "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib~=3.0->gradio) (1.16.0)\n",
591
+ "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich<14.0.0,>=10.11.0->typer[all]<1.0,>=0.9->gradio) (3.0.0)\n",
592
+ "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich<14.0.0,>=10.11.0->typer[all]<1.0,>=0.9->gradio) (2.16.1)\n",
593
+ "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich<14.0.0,>=10.11.0->typer[all]<1.0,>=0.9->gradio) (0.1.2)\n",
594
+ "Building wheels for collected packages: ffmpy\n",
595
+ " Building wheel for ffmpy (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
596
+ " Created wheel for ffmpy: filename=ffmpy-0.3.1-py3-none-any.whl size=5579 sha256=e32f9717874d35f7d4b17457be3439ecd7642e24c728fe5eb6e5d0bc480d2dd2\n",
597
+ " Stored in directory: /root/.cache/pip/wheels/01/a6/d1/1c0828c304a4283b2c1639a09ad86f83d7c487ef34c6b4a1bf\n",
598
+ "Successfully built ffmpy\n",
599
+ "Installing collected packages: pydub, ffmpy, websockets, typing-extensions, tomlkit, shellingham, semantic-version, python-multipart, orjson, h11, colorama, annotated-types, aiofiles, uvicorn, starlette, pydantic-core, httpcore, pydantic, httpx, gradio-client, fastapi, gradio\n",
600
+ " Attempting uninstall: typing-extensions\n",
601
+ " Found existing installation: typing_extensions 4.5.0\n",
602
+ " Uninstalling typing_extensions-4.5.0:\n",
603
+ " Successfully uninstalled typing_extensions-4.5.0\n",
604
+ " Attempting uninstall: pydantic\n",
605
+ " Found existing installation: pydantic 1.10.13\n",
606
+ " Uninstalling pydantic-1.10.13:\n",
607
+ " Successfully uninstalled pydantic-1.10.13\n",
608
+ "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
609
+ "lida 0.0.10 requires kaleido, which is not installed.\n",
610
+ "llmx 0.0.15a0 requires cohere, which is not installed.\n",
611
+ "llmx 0.0.15a0 requires openai, which is not installed.\n",
612
+ "llmx 0.0.15a0 requires tiktoken, which is not installed.\n",
613
+ "tensorflow-probability 0.22.0 requires typing-extensions<4.6.0, but you have typing-extensions 4.8.0 which is incompatible.\n",
614
+ "torchtext 0.16.0 requires torch==2.1.0, but you have torch 2.0.1+cu118 which is incompatible.\n",
615
+ "torchtext 0.16.0 requires torchdata==0.7.0, but you have torchdata 0.6.1 which is incompatible.\u001b[0m\u001b[31m\n",
616
+ "\u001b[0mSuccessfully installed aiofiles-23.2.1 annotated-types-0.6.0 colorama-0.4.6 fastapi-0.104.1 ffmpy-0.3.1 gradio-4.7.1 gradio-client-0.7.0 h11-0.14.0 httpcore-1.0.2 httpx-0.25.2 orjson-3.9.10 pydantic-2.5.2 pydantic-core-2.14.5 pydub-0.25.1 python-multipart-0.0.6 semantic-version-2.10.0 shellingham-1.5.4 starlette-0.27.0 tomlkit-0.12.0 typing-extensions-4.8.0 uvicorn-0.24.0.post1 websockets-11.0.3\n"
617
+ ]
618
+ }
619
+ ],
620
+ "source": [
621
+ "#@title Setup\n",
622
+ "/content\n",
623
+ "!nvidia-smi\n",
624
+ "!git clone https://github.com/Stability-AI/generative-models.git\n",
625
+ "# install required packages from pypi\n",
626
+ "# !pip3 install -r generative-models/requirements/pt2.txt\n",
627
+ "# manually install only necesarry packages for colab\n",
628
+ "!wget https://gist.githubusercontent.com/mkshing/4ad40699756d996ba6b3f7934e6ca532/raw/3f0094272c7a2bd3eb5f1a0db91bed582c9e8f01/requirements.txt\n",
629
+ "!pip3 install -r requirements.txt\n",
630
+ "!pip3 install -e generative-models\n",
631
+ "!pip3 install -e git+https://github.com/Stability-AI/datapipelines.git@main#egg=sdata\n",
632
+ "!pip3 install gradio"
633
+ ]
634
+ },
635
+ {
636
+ "cell_type": "code",
637
+ "execution_count": 2,
638
+ "metadata": {
639
+ "id": "zWlfaXvPbR1L"
640
+ },
641
+ "outputs": [],
642
+ "source": [
643
+ "#@title Colab hack for SVD\n",
644
+ "# !pip uninstall -y numpy\n",
645
+ "# !pip install -U numpy\n",
646
+ "!mkdir -p /content/scripts/util/detection\n",
647
+ "!ln -s /content/generative-models/scripts/util/detection/p_head_v1.npz /content/scripts/util/detection/p_head_v1.npz\n",
648
+ "!ln -s /content/generative-models/scripts/util/detection/w_head_v1.npz /content/scripts/util/detection/w_head_v1.npz"
649
+ ]
650
+ },
651
+ {
652
+ "cell_type": "code",
653
+ "execution_count": null,
654
+ "metadata": {
655
+ "cellView": "form",
656
+ "id": "v8O2yR3BLHv6",
657
+ "colab": {
658
+ "base_uri": "https://localhost:8080/"
659
+ },
660
+ "outputId": "a448e8cd-955f-40b9-960d-c84cf6e7ce32"
661
+ },
662
+ "outputs": [
663
+ {
664
+ "output_type": "stream",
665
+ "name": "stdout",
666
+ "text": [
667
+ "download from https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt/resolve/main/svd_xt.safetensors to checkpoints/svd_xt.safetensors\n"
668
+ ]
669
+ }
670
+ ],
671
+ "source": [
672
+ "# @title Download weights\n",
673
+ "import os\n",
674
+ "import subprocess\n",
675
+ "version = \"svd_xt\" #@param [\"svd\", \"svd_xt\"]\n",
676
+ "TYPE2PATH = {\n",
677
+ " \"svd\": [\"https://huggingface.co/stabilityai/stable-video-diffusion-img2vid/resolve/main/svd.safetensors\", \"checkpoints/svd.safetensors\"],\n",
678
+ " \"svd_xt\": [\"https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt/resolve/main/svd_xt.safetensors\", \"checkpoints/svd_xt.safetensors\"]\n",
679
+ "}\n",
680
+ "download_from, download_to = TYPE2PATH[version]\n",
681
+ "# @markdown This will take several minutes. <br>\n",
682
+ "# @markdown **Reference:**\n",
683
+ "# @markdown * `svd`: [stabilityai/stable-video-diffusion-img2vid](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid) for 14 frames generation\n",
684
+ "# @markdown * `svd_xt`: [stabilityai/stable-video-diffusion-img2vid-xt](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt) for 25 frames generation\n",
685
+ "os.makedirs(\"checkpoints\", exist_ok=True)\n",
686
+ "if os.path.exists(download_to):\n",
687
+ " print(\"Already downloaded\")\n",
688
+ "else:\n",
689
+ " print(f\"download from {download_from} to {download_to}\")\n",
690
+ " subprocess.call([\"wget\", download_from, \"-O\", download_to])"
691
+ ]
692
+ },
693
+ {
694
+ "cell_type": "code",
695
+ "execution_count": null,
696
+ "metadata": {
697
+ "id": "9AZDrh-SUDt2"
698
+ },
699
+ "outputs": [],
700
+ "source": [
701
+ "#@title Load Model\n",
702
+ "import sys\n",
703
+ "from omegaconf import OmegaConf\n",
704
+ "\n",
705
+ "import torch\n",
706
+ "\n",
707
+ "sys.path.append(\"generative-models\")\n",
708
+ "from sgm.util import default, instantiate_from_config\n",
709
+ "from scripts.util.detection.nsfw_and_watermark_dectection import DeepFloydDataFiltering\n",
710
+ "\n",
711
+ "def load_model(\n",
712
+ " config: str,\n",
713
+ " device: str,\n",
714
+ " num_frames: int,\n",
715
+ " num_steps: int,\n",
716
+ "):\n",
717
+ " config = OmegaConf.load(config)\n",
718
+ " config.model.params.conditioner_config.params.emb_models[\n",
719
+ " 0\n",
720
+ " ].params.open_clip_embedding_config.params.init_device = device\n",
721
+ " config.model.params.sampler_config.params.num_steps = num_steps\n",
722
+ " config.model.params.sampler_config.params.guider_config.params.num_frames = (\n",
723
+ " num_frames\n",
724
+ " )\n",
725
+ " with torch.device(device):\n",
726
+ " model = instantiate_from_config(config.model).to(device).eval().requires_grad_(False)\n",
727
+ "\n",
728
+ " filter = DeepFloydDataFiltering(verbose=False, device=device)\n",
729
+ " return model, filter\n",
730
+ "\n",
731
+ "\n",
732
+ "if version == \"svd\":\n",
733
+ " num_frames = 14\n",
734
+ " num_steps = 25\n",
735
+ " # output_folder = default(output_folder, \"outputs/simple_video_sample/svd/\")\n",
736
+ " model_config = \"generative-models/scripts/sampling/configs/svd.yaml\"\n",
737
+ "elif version == \"svd_xt\":\n",
738
+ " num_frames = 25\n",
739
+ " num_steps = 30\n",
740
+ " # output_folder = default(output_folder, \"outputs/simple_video_sample/svd_xt/\")\n",
741
+ " model_config = \"generative-models/scripts/sampling/configs/svd_xt.yaml\"\n",
742
+ "else:\n",
743
+ " raise ValueError(f\"Version {version} does not exist.\")\n",
744
+ "\n",
745
+ "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
746
+ "model, filter = load_model(\n",
747
+ " model_config,\n",
748
+ " device,\n",
749
+ " num_frames,\n",
750
+ " num_steps,\n",
751
+ ")\n",
752
+ "# move models expect unet to cpu\n",
753
+ "model.conditioner.cpu()\n",
754
+ "model.first_stage_model.cpu()\n",
755
+ "# change the dtype of unet\n",
756
+ "model.model.to(dtype=torch.float16)\n",
757
+ "torch.cuda.empty_cache()\n",
758
+ "model = model.requires_grad_(False)"
759
+ ]
760
+ },
761
+ {
762
+ "cell_type": "code",
763
+ "execution_count": null,
764
+ "metadata": {
765
+ "id": "x1-dnq0RT95O"
766
+ },
767
+ "outputs": [],
768
+ "source": [
769
+ "# @title Sampling function\n",
770
+ "import math\n",
771
+ "import os\n",
772
+ "from glob import glob\n",
773
+ "from pathlib import Path\n",
774
+ "from typing import Optional\n",
775
+ "\n",
776
+ "import cv2\n",
777
+ "import numpy as np\n",
778
+ "import torch\n",
779
+ "from einops import rearrange, repeat\n",
780
+ "from fire import Fire\n",
781
+ "\n",
782
+ "from PIL import Image\n",
783
+ "from torchvision.transforms import ToTensor\n",
784
+ "from torchvision.transforms import functional as TF\n",
785
+ "\n",
786
+ "from sgm.inference.helpers import embed_watermark\n",
787
+ "from sgm.util import default, instantiate_from_config\n",
788
+ "\n",
789
+ "\n",
790
+ "def get_unique_embedder_keys_from_conditioner(conditioner):\n",
791
+ " return list(set([x.input_key for x in conditioner.embedders]))\n",
792
+ "\n",
793
+ "\n",
794
+ "def get_batch(keys, value_dict, N, T, device, dtype=None):\n",
795
+ " batch = {}\n",
796
+ " batch_uc = {}\n",
797
+ "\n",
798
+ " for key in keys:\n",
799
+ " if key == \"fps_id\":\n",
800
+ " batch[key] = (\n",
801
+ " torch.tensor([value_dict[\"fps_id\"]])\n",
802
+ " .to(device, dtype=dtype)\n",
803
+ " .repeat(int(math.prod(N)))\n",
804
+ " )\n",
805
+ " elif key == \"motion_bucket_id\":\n",
806
+ " batch[key] = (\n",
807
+ " torch.tensor([value_dict[\"motion_bucket_id\"]])\n",
808
+ " .to(device, dtype=dtype)\n",
809
+ " .repeat(int(math.prod(N)))\n",
810
+ " )\n",
811
+ " elif key == \"cond_aug\":\n",
812
+ " batch[key] = repeat(\n",
813
+ " torch.tensor([value_dict[\"cond_aug\"]]).to(device, dtype=dtype),\n",
814
+ " \"1 -> b\",\n",
815
+ " b=math.prod(N),\n",
816
+ " )\n",
817
+ " elif key == \"cond_frames\":\n",
818
+ " batch[key] = repeat(value_dict[\"cond_frames\"], \"1 ... -> b ...\", b=N[0])\n",
819
+ " elif key == \"cond_frames_without_noise\":\n",
820
+ " batch[key] = repeat(\n",
821
+ " value_dict[\"cond_frames_without_noise\"], \"1 ... -> b ...\", b=N[0]\n",
822
+ " )\n",
823
+ " else:\n",
824
+ " batch[key] = value_dict[key]\n",
825
+ "\n",
826
+ " if T is not None:\n",
827
+ " batch[\"num_video_frames\"] = T\n",
828
+ "\n",
829
+ " for key in batch.keys():\n",
830
+ " if key not in batch_uc and isinstance(batch[key], torch.Tensor):\n",
831
+ " batch_uc[key] = torch.clone(batch[key])\n",
832
+ " return batch, batch_uc\n",
833
+ "\n",
834
+ "\n",
835
+ "\n",
836
+ "def sample(\n",
837
+ " input_path: str = \"assets/test_image.png\", # Can either be image file or folder with image files\n",
838
+ " resize_image: bool = False,\n",
839
+ " num_frames: Optional[int] = None,\n",
840
+ " num_steps: Optional[int] = None,\n",
841
+ " fps_id: int = 6,\n",
842
+ " motion_bucket_id: int = 127,\n",
843
+ " cond_aug: float = 0.02,\n",
844
+ " seed: int = 23,\n",
845
+ " decoding_t: int = 14, # Number of frames decoded at a time! This eats most VRAM. Reduce if necessary.\n",
846
+ " device: str = \"cuda\",\n",
847
+ " output_folder: Optional[str] = \"/content/outputs\",\n",
848
+ " skip_filter: bool = False,\n",
849
+ "):\n",
850
+ " \"\"\"\n",
851
+ " Simple script to generate a single sample conditioned on an image `input_path` or multiple images, one for each\n",
852
+ " image file in folder `input_path`. If you run out of VRAM, try decreasing `decoding_t`.\n",
853
+ " \"\"\"\n",
854
+ " torch.manual_seed(seed)\n",
855
+ "\n",
856
+ " path = Path(input_path)\n",
857
+ " all_img_paths = []\n",
858
+ " if path.is_file():\n",
859
+ " if any([input_path.endswith(x) for x in [\"jpg\", \"jpeg\", \"png\"]]):\n",
860
+ " all_img_paths = [input_path]\n",
861
+ " else:\n",
862
+ " raise ValueError(\"Path is not valid image file.\")\n",
863
+ " elif path.is_dir():\n",
864
+ " all_img_paths = sorted(\n",
865
+ " [\n",
866
+ " f\n",
867
+ " for f in path.iterdir()\n",
868
+ " if f.is_file() and f.suffix.lower() in [\".jpg\", \".jpeg\", \".png\"]\n",
869
+ " ]\n",
870
+ " )\n",
871
+ " if len(all_img_paths) == 0:\n",
872
+ " raise ValueError(\"Folder does not contain any images.\")\n",
873
+ " else:\n",
874
+ " raise ValueError\n",
875
+ " all_out_paths = []\n",
876
+ " for input_img_path in all_img_paths:\n",
877
+ " with Image.open(input_img_path) as image:\n",
878
+ " if image.mode == \"RGBA\":\n",
879
+ " image = image.convert(\"RGB\")\n",
880
+ " if resize_image and image.size != (1024, 576):\n",
881
+ " print(f\"Resizing {image.size} to (1024, 576)\")\n",
882
+ " image = TF.resize(TF.resize(image, 1024), (576, 1024))\n",
883
+ " w, h = image.size\n",
884
+ "\n",
885
+ " if h % 64 != 0 or w % 64 != 0:\n",
886
+ " width, height = map(lambda x: x - x % 64, (w, h))\n",
887
+ " image = image.resize((width, height))\n",
888
+ " print(\n",
889
+ " f\"WARNING: Your image is of size {h}x{w} which is not divisible by 64. We are resizing to {height}x{width}!\"\n",
890
+ " )\n",
891
+ "\n",
892
+ " image = ToTensor()(image)\n",
893
+ " image = image * 2.0 - 1.0\n",
894
+ "\n",
895
+ " image = image.unsqueeze(0).to(device)\n",
896
+ " H, W = image.shape[2:]\n",
897
+ " assert image.shape[1] == 3\n",
898
+ " F = 8\n",
899
+ " C = 4\n",
900
+ " shape = (num_frames, C, H // F, W // F)\n",
901
+ " if (H, W) != (576, 1024):\n",
902
+ " print(\n",
903
+ " \"WARNING: The conditioning frame you provided is not 576x1024. This leads to suboptimal performance as model was only trained on 576x1024. Consider increasing `cond_aug`.\"\n",
904
+ " )\n",
905
+ " if motion_bucket_id > 255:\n",
906
+ " print(\n",
907
+ " \"WARNING: High motion bucket! This may lead to suboptimal performance.\"\n",
908
+ " )\n",
909
+ "\n",
910
+ " if fps_id < 5:\n",
911
+ " print(\"WARNING: Small fps value! This may lead to suboptimal performance.\")\n",
912
+ "\n",
913
+ " if fps_id > 30:\n",
914
+ " print(\"WARNING: Large fps value! This may lead to suboptimal performance.\")\n",
915
+ "\n",
916
+ " value_dict = {}\n",
917
+ " value_dict[\"motion_bucket_id\"] = motion_bucket_id\n",
918
+ " value_dict[\"fps_id\"] = fps_id\n",
919
+ " value_dict[\"cond_aug\"] = cond_aug\n",
920
+ " value_dict[\"cond_frames_without_noise\"] = image\n",
921
+ " value_dict[\"cond_frames\"] = image + cond_aug * torch.randn_like(image)\n",
922
+ " value_dict[\"cond_aug\"] = cond_aug\n",
923
+ " # low vram mode\n",
924
+ " model.conditioner.cpu()\n",
925
+ " model.first_stage_model.cpu()\n",
926
+ " torch.cuda.empty_cache()\n",
927
+ " model.sampler.verbose = True\n",
928
+ "\n",
929
+ " with torch.no_grad():\n",
930
+ " with torch.autocast(device):\n",
931
+ " model.conditioner.to(device)\n",
932
+ " batch, batch_uc = get_batch(\n",
933
+ " get_unique_embedder_keys_from_conditioner(model.conditioner),\n",
934
+ " value_dict,\n",
935
+ " [1, num_frames],\n",
936
+ " T=num_frames,\n",
937
+ " device=device,\n",
938
+ " )\n",
939
+ " c, uc = model.conditioner.get_unconditional_conditioning(\n",
940
+ " batch,\n",
941
+ " batch_uc=batch_uc,\n",
942
+ " force_uc_zero_embeddings=[\n",
943
+ " \"cond_frames\",\n",
944
+ " \"cond_frames_without_noise\",\n",
945
+ " ],\n",
946
+ " )\n",
947
+ " model.conditioner.cpu()\n",
948
+ " torch.cuda.empty_cache()\n",
949
+ "\n",
950
+ " # from here, dtype is fp16\n",
951
+ " for k in [\"crossattn\", \"concat\"]:\n",
952
+ " uc[k] = repeat(uc[k], \"b ... -> b t ...\", t=num_frames)\n",
953
+ " uc[k] = rearrange(uc[k], \"b t ... -> (b t) ...\", t=num_frames)\n",
954
+ " c[k] = repeat(c[k], \"b ... -> b t ...\", t=num_frames)\n",
955
+ " c[k] = rearrange(c[k], \"b t ... -> (b t) ...\", t=num_frames)\n",
956
+ " for k in uc.keys():\n",
957
+ " uc[k] = uc[k].to(dtype=torch.float16)\n",
958
+ " c[k] = c[k].to(dtype=torch.float16)\n",
959
+ "\n",
960
+ " randn = torch.randn(shape, device=device, dtype=torch.float16)\n",
961
+ "\n",
962
+ " additional_model_inputs = {}\n",
963
+ " additional_model_inputs[\"image_only_indicator\"] = torch.zeros(\n",
964
+ " 2, num_frames\n",
965
+ " ).to(device, )\n",
966
+ " additional_model_inputs[\"num_video_frames\"] = batch[\"num_video_frames\"]\n",
967
+ "\n",
968
+ " for k in additional_model_inputs:\n",
969
+ " if isinstance(additional_model_inputs[k], torch.Tensor):\n",
970
+ " additional_model_inputs[k] = additional_model_inputs[k].to(dtype=torch.float16)\n",
971
+ "\n",
972
+ " def denoiser(input, sigma, c):\n",
973
+ " return model.denoiser(\n",
974
+ " model.model, input, sigma, c, **additional_model_inputs\n",
975
+ " )\n",
976
+ "\n",
977
+ " samples_z = model.sampler(denoiser, randn, cond=c, uc=uc)\n",
978
+ " samples_z.to(dtype=model.first_stage_model.dtype)\n",
979
+ " ##\n",
980
+ "\n",
981
+ " model.en_and_decode_n_samples_a_time = decoding_t\n",
982
+ " model.first_stage_model.to(device)\n",
983
+ " samples_x = model.decode_first_stage(samples_z)\n",
984
+ " samples = torch.clamp((samples_x + 1.0) / 2.0, min=0.0, max=1.0)\n",
985
+ " model.first_stage_model.cpu()\n",
986
+ " torch.cuda.empty_cache()\n",
987
+ "\n",
988
+ " os.makedirs(output_folder, exist_ok=True)\n",
989
+ " base_count = len(glob(os.path.join(output_folder, \"*.mp4\")))\n",
990
+ " video_path = os.path.join(output_folder, f\"{base_count:06d}.mp4\")\n",
991
+ " writer = cv2.VideoWriter(\n",
992
+ " video_path,\n",
993
+ " cv2.VideoWriter_fourcc(*\"MP4V\"),\n",
994
+ " fps_id + 1,\n",
995
+ " (samples.shape[-1], samples.shape[-2]),\n",
996
+ " )\n",
997
+ "\n",
998
+ " samples = embed_watermark(samples)\n",
999
+ " if not skip_filter:\n",
1000
+ " samples = filter(samples)\n",
1001
+ " else:\n",
1002
+ " print(\"WARNING: You have disabled the NSFW/Watermark filter. Please do not expose unfiltered results in services or applications open to the public.\")\n",
1003
+ " vid = (\n",
1004
+ " (rearrange(samples, \"t c h w -> t h w c\") * 255)\n",
1005
+ " .cpu()\n",
1006
+ " .numpy()\n",
1007
+ " .astype(np.uint8)\n",
1008
+ " )\n",
1009
+ " for frame in vid:\n",
1010
+ " frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)\n",
1011
+ " writer.write(frame)\n",
1012
+ " writer.release()\n",
1013
+ " all_out_paths.append(video_path)\n",
1014
+ " return all_out_paths\n",
1015
+ "\n"
1016
+ ]
1017
+ },
1018
+ {
1019
+ "cell_type": "code",
1020
+ "execution_count": null,
1021
+ "metadata": {
1022
+ "id": "5MdVILPlMUDe"
1023
+ },
1024
+ "outputs": [],
1025
+ "source": [
1026
+ "# @title Do the Run! markdown Generation takes about 10 mins for `svd_xt` on T4 (Colab free plan). Please be patient...\n",
1027
+ "import gradio as gr\n",
1028
+ "import random\n",
1029
+ "\n",
1030
+ "\n",
1031
+ "def infer(input_path: str, resize_image: bool, n_frames: int, n_steps: int, seed: str, decoding_t: int, fps_id: int, motion_bucket_id: int, cond_aug: float, skip_filter: bool = False) -> str:\n",
1032
+ " if seed == \"random\":\n",
1033
+ " seed = random.randint(0, 2**32)\n",
1034
+ " seed = int(seed)\n",
1035
+ " output_paths = sample(\n",
1036
+ " input_path=input_path,\n",
1037
+ " resize_image=resize_image,\n",
1038
+ " num_frames=n_frames,\n",
1039
+ " num_steps=n_steps,\n",
1040
+ " fps_id=fps_id,\n",
1041
+ " motion_bucket_id=motion_bucket_id,\n",
1042
+ " cond_aug=cond_aug,\n",
1043
+ " seed=seed,\n",
1044
+ " decoding_t=decoding_t, # Number of frames decoded at a time! This eats most VRAM. Reduce if necessary.\n",
1045
+ " device=device,\n",
1046
+ " skip_filter=skip_filter,\n",
1047
+ " )\n",
1048
+ " return output_paths[0]\n",
1049
+ "\n",
1050
+ "\n",
1051
+ "with gr.Blocks() as demo:\n",
1052
+ " with gr.Column():\n",
1053
+ " image = gr.Image(label=\"input image\", type=\"filepath\")\n",
1054
+ " resize_image = gr.Checkbox(label=\"resize to optimal size\", value=True)\n",
1055
+ " btn = gr.Button(\"Run\")\n",
1056
+ " with gr.Accordion(label=\"Advanced options\", open=False):\n",
1057
+ " n_frames = gr.Number(precision=0, label=\"number of frames\", value=num_frames)\n",
1058
+ " n_steps = gr.Number(precision=0, label=\"number of steps\", value=num_steps)\n",
1059
+ " seed = gr.Text(value=\"random\", label=\"seed (integer or 'random')\",)\n",
1060
+ " decoding_t = gr.Number(precision=0, label=\"number of frames decoded at a time\", value=2)\n",
1061
+ " fps_id = gr.Number(precision=0, label=\"frames per second\", value=6)\n",
1062
+ " motion_bucket_id = gr.Number(precision=0, value=127, label=\"motion bucket id\")\n",
1063
+ " cond_aug = gr.Number(label=\"condition augmentation factor\", value=0.02)\n",
1064
+ " skip_filter = gr.Checkbox(value=False, label=\"skip nsfw/watermark filter\")\n",
1065
+ " with gr.Column():\n",
1066
+ " video_out = gr.Video(label=\"generated video\")\n",
1067
+ " examples = [\n",
1068
+ " [\"https://user-images.githubusercontent.com/33302880/284758167-367a25d8-8d7b-42d3-8391-6d82813c7b0f.png\"]\n",
1069
+ " ]\n",
1070
+ " inputs = [image, resize_image, n_frames, n_steps, seed, decoding_t, fps_id, motion_bucket_id, cond_aug, skip_filter]\n",
1071
+ " outputs = [video_out]\n",
1072
+ " btn.click(infer, inputs=inputs, outputs=outputs)\n",
1073
+ " gr.Examples(examples=examples, inputs=inputs, outputs=outputs, fn=infer)\n",
1074
+ " demo.queue().launch(debug=True, share=True, show_error=True)"
1075
+ ]
1076
+ }
1077
+ ],
1078
+ "metadata": {
1079
+ "accelerator": "GPU",
1080
+ "colab": {
1081
+ "gpuType": "T4",
1082
+ "provenance": []
1083
+ },
1084
+ "kernelspec": {
1085
+ "display_name": "Python 3",
1086
+ "name": "python3"
1087
+ },
1088
+ "language_info": {
1089
+ "name": "python"
1090
+ }
1091
+ },
1092
+ "nbformat": 4,
1093
+ "nbformat_minor": 0
1094
+ }