diff --git a/.gitattributes b/.gitattributes index a6344aac8c09253b3b630fb776ae94478aa0275b..d7e23687df4ace544c9390d221cac33394edbab0 100644 --- a/.gitattributes +++ b/.gitattributes @@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text *.zip filter=lfs diff=lfs merge=lfs -text *.zst filter=lfs diff=lfs merge=lfs -text *tfevents* filter=lfs diff=lfs merge=lfs -text +AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/all_in_one.png filter=lfs diff=lfs merge=lfs -text +AiAF/Backup-SD-Extensions-Folder/sd-civitai-browser-plus-1.15.2/aria2/lin/aria2 filter=lfs diff=lfs merge=lfs -text +AiAF/Backup-SD-Extensions-Folder/sd-civitai-browser-plus-1.15.2/aria2/win/aria2.exe filter=lfs diff=lfs merge=lfs -text +AiAF/Backup-SD-Extensions-Folder/sd-dynamic-prompts/images/weighting-colours.png filter=lfs diff=lfs merge=lfs -text +AiAF/Backup-SD-Extensions-Folder/sd-dynamic-prompts/images/weighting-us-population.png filter=lfs diff=lfs merge=lfs -text diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/.github/ISSUE_TEMPLATE/simple-issue-template.md b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/.github/ISSUE_TEMPLATE/simple-issue-template.md new file mode 100644 index 0000000000000000000000000000000000000000..82fb14ed9f2ef9ebc6f6bba8cb8217d07a7454d7 --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/.github/ISSUE_TEMPLATE/simple-issue-template.md @@ -0,0 +1,20 @@ +--- +name: Simple Issue template +about: Describe this issue template's purpose here. +title: '' +labels: '' +assignees: '' + +--- + +## Have you read document? + +## Have you checked console log window's msg? + +## Describe Issue + + +## Screenshot for UI issue + + +## Console log's msg or screenshot for function issue diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/.gitignore b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/.gitignore new file mode 100644 index 0000000000000000000000000000000000000000..88e0fd7f7075b7d7e5d567a497db384b9ce49a3d --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/.gitignore @@ -0,0 +1,4 @@ +scripts/__pycache__/ +scripts/ch_lib/__pycache__/ +setting.json +**/*.kate-swp diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/CHANGELOG.md b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/CHANGELOG.md new file mode 100644 index 0000000000000000000000000000000000000000..d27eeb238b159eadece7356c25bc2c558e2f256f --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/CHANGELOG.md @@ -0,0 +1,148 @@ +# Change Log +## 1.7.7 +* Implemented authentication via API Key. + * This allows the user to download models that require authentication. + +## 1.7.6 +* Downloader rewrite: + * Downloading models will now show download progress in webui. + * Download errors will now be displayed in webui, with more detailed information still in console output. +* The current model filename will now be included when renaming models. +* Extension should now at least run in [Vladmandic's stable diffusion webui](https://github.com/vladmandic/automatic) thanks to [#32](https://github.com/zixaphir/Stable-Diffusion-Webui-Civitai-Helper/pull/32). + * I do not run Vladmandic and have not personally tested functionality. Users who would like to address issues with Vladmandic compatibility may send a [Pull Request](https://github.com/zixaphir/Stable-Diffusion-Webui-Civitai-Helper/pulls). + +## 1.7.5 +* Missing image generation info in civitai.info files can now be retrieved from civitai by downloading the image and parsing its EXIF data. Most people should not need this, but it can be helpful if you're trying to recreate an image or more like it from the model's preview page. This will only work if the model author left metadata in the image. +* Added some compatibility code for the Lobe theme extension +* Reduce timeout errors, retry failed downloads. +* Added an option to use sd webui's SHA256 hashing function instead of our own. + +## 1.7.4 +* Added "Remove Model" button. From butaixianran's repo. +* Added "Rename Model" button. +* New `.json` files will now have their descriptions surrounded by HTML comments as to not break model cards when card descriptions are enabled. If you want to display these descriptions on cards, you can remove the leading ``, but you will be responsible for any issues that arise. +* Card buttons can now be hidden individually. By default, the "insert trigger words" button is hidden on newer versions of sdwebui, where this functionality happens automatically if the metadata `JSON` exists, which is created when models are scanned with default settings. +* Model names on cards now have a maximum height when not hovered, which should prevent the model name from occupying the whole card. +* Fix downloading size check issue. From butaixianran's repo. +* NSFW previews can now be blocked by severity, from Soft, Mature, X, and Allow All. From sdy623's repo. These don't seem to be documented on Civitai's end, but here's what I've been able to gather by pinging the API and seeing what images were returned. + * Soft images do not appear to contain explicit nudity, but may have implicitly suggestive elements or clothing that outline the underlying body closely enough that details underneath poke out or have their silhoutte visible. + * Mature images may contain nudity, but could just as easily show a little too much cleavage or are otherwise actively sexually suggestive. + * X are explicitly pornographic. + +## v1.7.3 +* Downloading webui model information (`[model_name].json`) is now optional and can be configured in the webui settings. +* When a model is not found on Civitai, this extension will attempt to read the model header for activation keywords. This only works with safetensors files with kohya_ss metadata and may be inaccurate depending on how the model was trained. This method assumes the model was trained with each trained concept having its own training folder. This is not always the case. + * This information can also be found in webui without any extensions by clicking the :information_source: button and reading the `"ss_tag_frequency":` section. This section may prove more helpful on models that are trained on multiple concepts but with all the training data under one directory. For instance, if it was trained for certain characters, search the list for that character's name. Often model authors will use a name plus a unique identifier as to not cause the character to "bleed into" other characters with the same name. For instance, we can the [All The Mothers](https://civitai.com/models/48200) lora has all of its trained characters as tagged images under one dataset: + +![](img/all_the_mothers.png) + +## v1.7.2 +* Better HTML sanitization/removal from Descrition/Note fields. +* CivitAI Anti-DDOS false-positive delay lowered. This may be reverted if it causes issues, tho <5 API hits a second shouldn't be too bad. +* CivitAI Anti-DDOS false-positive delay added to all models, not just embeddings, when performing model scans. + * However, delay has been removed when a scanned model does not need to perform an API hit to CivitAI. +* Enabled functionality to re-download model data if the current data is from an older version of SD-Webui-Civitai-Helper. + * For many people, this will not be necessary. For those migrating from the original extension, missing data for WebUI is handled separately and will be downloaded on scan anyways. + * For more details, see the [wiki](https://github.com/zixaphir/Stable-Diffusion-Webui-Civitai-Helper/wiki/Metadata-Format-Changes). +* **HOTFIX**: if a model has been deleted from Civitai, the `Replace Old Metadata Formats` option will no longer overwrite working data with dummy data. + +## v1.7.1 +* Settings are now in the settings menu. +* Model previews following the name format `[model name].preview.[format]` in JPEG, GIF, and WEBP formats are now checked before downloading model previews from civitai. +* HTML in the model description and version information should be removed and replaced more eloquently. +* Improved layout of Model Update interface. +* Added options for behaviors when downloading a model with the same filename as one already existing. + * The new default is to skip downloading, replacing the old default of renaming the new model. +* **Post-release Hotfix**: fixed skeleton JSON generation for models not found on CivitAI. + +## v1.7.0 +* Added compatibility with Stable Diffusion WebUI v1.5.0-v1.6.0. +* Downloading metadata from Civitai should now include more exhaustive data, including correct description and tags. +* NSFW preview detection should be accurate. +* Native and Extension LyCoris support should function. +* Buttons added by this extension no longer require a separate button to add to UI. +* Extension populates Stable Diffusion WebUI's new metadata editor with description, Checkpoint model version, version information, and activation prompt(s). + * This should allow simply clicking a card to add model activation prompts. +* Model Download UI requires fewer clicks to download a model. + +## v1.6.4 +* Add "Download All files" checkbox for downloading model section. Uncheck means only download 1 file. + +## v1.6.3 +* Support downloading multiple files, not avaiable when checking new version. + +## v1.6.2.1 +* when parsing civitai url, remove query string by PR + +## v1.6.2 +* When downloading, re-name file if file already exists + +## v1.6.1.1 +* Support bilingual localization extension by PR + +## v1.6.1 +* Fix Localization issue for 4 addtional buttons on cards. (Forgot that again...) + +## v1.6.0 +* Fix some UI issues to work with gradio 3.23.0 +* Support Proxy when connecting to civitai. Check document for detail. +* check realpath when opening file, to fix error when using junction +* Fix multiple addtional buttons issue after switching tabs. + +## v1.5.7 +* Fix Localization issue for 4 addtional buttons on cards + +## v1.5.6 +* update error msg when can not connect to civitai API service +* update thumb mode for SD webui new version's metadata button + +## v1.5.5 +* update SHA256 function, now it just use the code from pip + +## v1.5.4 +* set sys.stdout to utf-8 +* Add default header for requests to prevent from being blocked by civitai. +* merge other v1.5.x change log to v1.5.4 +* When downloading a model by url, check if target model version is already existed in user selected sub-folder. +* Support scanning only selected model types. +* Force TI scanning delay 1 second to prevent from civitai treating this extension's requests as attacking. + +## v1.5.0 +* Download a model by Civitai model page's url +* Resume downloading from break-point +* Download new version into SD Webui's model folder +* Addtional button now works on thumbnail mode +* Option to always show addtion button, for touch screen. + +## v1.4.2 +* ignore .vae file in model folder when scanning + +## v1.4.1 +* When checking new versions, also searching and ignore already existed ones. +* Add version number to the bottom of this extension's tab + +## v1.4 +* Support checking model's new version, display the result in UI and offer download url +* Remove addintional sub tabs on extension tab. make ui simpler. + +## v1.3 +* Open url at client side +* Link selected model to civitai by url or model id +* Save and load extension setting to file +* Show button action's output to UI +* Code refactoring + +## v1.2.1 +* Add more error checking to work with different versions of SD webui. + +## v1.2 +* Support customer model folder +* Support readable model info file +* Support download preview image with max size +* Remove card buttons when extra network is in thumbnail mode + +## v1.1 +* Support subfolders +* Check if refresh is needed when clicking "Refresh Civitai Helper" +* Add space when adding trigger words +* Add memory Optimized sha256 as an option diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/README.cn.md b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/README.cn.md new file mode 100644 index 0000000000000000000000000000000000000000..3e95f43d7d11009cb64ab1392ff8ece0fd5568eb --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/README.cn.md @@ -0,0 +1,251 @@ +# 这份文件已由ChatGPT从英语翻译而来。 + +**[贡献者](https://github.com/zixaphir/Stable-Diffusion-Webui-Civitai-Helper/graphs/contributors)** + +##关于Civitai助手 +这个扩展提供了从Civitai下载模型和模型元数据的能力。您可以轻松获得托管在Civitai上的模型的激活关键词、模型描述、版本信息以及模型预览等数据,而无需离开稳定的扩散WebUI。 + +##关于Civitai助手的这个版本 +这是我个人版本的Stable-Diffusion-Webui-Civitai-Helper。我完全是因为我使用的版本在SD-webui v1.5发布时崩溃了,我需要它能够正常工作,所以开始了这个项目。从那时起,我添加了我想要的功能,并尽力维护与较旧版本的sd-webui的兼容性,但我不使用较旧版本,因此不会在较旧版本上进行测试。 + +我打算尽我所能使其保持正常运行,但这只是一个业余项目,我远没有其他更有经验的扩展开发者那么娴熟。我可能会长时间离开,如果失去兴趣甚至可能永远离开。我容易犯错,也可能会出现错误和漏洞。 + +如果有更有兴趣的人决定接手开发,我将乐意放弃对这个项目的控制,并尝试将我的更改推送到他们的项目中,如果有需要的话。 + + +# Civitai Helper +Stable Diffusion Webui 扩展Civitai助手,用于更轻松的管理和使用Civitai模型。 + +[Civitai Url](https://civitai.com/models/16768/civitai-helper-sd-webui-civitai-extension) + +# 注意 +**本插件现在非常稳定,很多人用得很好,如果碰到问题,先看[常见问题](#常见问题),并检查命令行窗口的详情。** +开issue前,请先看文档。找茬行为的用户将会被拉黑,参考:[找茬行为会被拉黑](https://github.com/butaixianran/Stable-Diffusion-Webui-Civitai-Helper/issues/96#issuecomment-1500310981) + + +# 功能 +[中文介绍视频(非官方)](https://youtu.be/x4tPWPmeAgM?t=373) + +* 扫描所有模型,从Civitai下载模型信息和预览图 +* 通过civitai模型页面url,连接本地模型和civitai模型信息 +* 通过Civitai模型页面url,下载模型(含信息和预览图)到SD目录或子目录。 +* 下载支持断点续传 +* 批量检查本地模型,在civitai上的新版本 +* 直接下载新版本模型到SD模型目录内(含信息和预览图) +* 修改了内置的"Extra Network"模型卡片,每个卡片增加了如下功能按钮: + - 🖼: 修改文字"replace preview"为这个图标 + - 🌐: 在新标签页打开这个模型的Civitai页面 + - 💡: 一键添加这个模型的触发词到关键词输入框 + - 🏷: 一键使用这个模型预览图所使用的关键词 +* 以上额外功能按钮支持thumbnail模式 +* 增加一直显示按钮的选项,以供触屏用户使用 + + +# 安装 +下载本项目为zip文件,解压到`你的SD webui目录/extensions`下即可。 + +不管是安装还是升级本插件,都要整个关闭SD Webui,重新启动它。只是Reload UI不起作用。 + +(如果用SD webui的插件界面安装,请先给git配置代理。它不是通过浏览器下载,是通过git下载。) + + +# 使用方法 + +## 更新你的SD webui +本扩展需要取到 Extra Network的卡片列表id。**这个是2023-02-06,才添加到SD webui里面的。** + +所以,如果你用的版本比这个早,你就需要先更新你的SD Webui! + + +## 扫描模型 +前往扩展页面"Civitai Helper",有个按钮叫:"Scan Model" + +![](img/extension_tab.jpg) + +点击,就会扫描所有模型,生成SHA256码,用于从civitai获取模型信息和预览图。**扫描需要很久,耐心等待**。 + +每个模型,本扩展都会创建一个json文件,用来保存从civitai得到的模型信息。这个文件会保存在模型同目录下,名称为:"模型名字.civitai.info"。 + +![](img/model_info_file.jpg) + +如果模型信息文件已经存在,扫描时就会跳过这个模型。如果模型不是civitai的,就会创建个空信息文件,以避免以后重复扫描。 + +### 添加新模型 +当你下载了新模型之后,只要再次点击扫描按钮即可。已经扫描过的文件不会重复扫描,会自动得到新模型的信息和预览图。无须重启SD webui。 + +## 模型卡片 +**(先完成扫描,再使用卡片功能)** +打开SD webui's 内置的 "Extra Network" 页面,显示模型卡片 + +![](img/extra_network.jpg) + + +移动鼠标到模型卡片底部,就会显示4个按钮: + - 🖼: 修改文字"replace preview"为这个图标 + - 🌐: 在新标签页打开这个模型的Civitai页面 + - 💡: 一键添加这个模型的触发词到关键词输入框 + - 🏷: 一键使用这个模型预览图所使用的关键词 + +![](img/model_card.jpg) + +如果你没有看到这些额外的按钮,只要点击`Refresh Civitai Helper`,他们就会被重新添加到卡片上。 + +![](img/refresh_ch.jpg) + +每次当Extra Network刷新,他都会删除掉额外的修改,我们的按钮就会消失。这时你就需要点击`Refresh Civitai Helper`把这些功能添加回去。 + + +### 小图模式 +以上功能按钮支持小图模式,但受制于SD Webui的CSS问题,目前,只能要么一直显示,要么一直不显示,不能鼠标滑过才显示。 +![](img/thumb_mode.jpg) + +## 下载 +**(单任务,下载完一个再下另一个)** +通过Civitai模型页面Url下载模型,要3个步骤: +* 填入url,点击按钮获取模型信息 +* 扩展会自动填入模型名称和类型,你需要选择下载的子目录和模型版本。 +* 点击下载 +![](img/download_model.jpg) + +下载过程会显示在命令行界面带个进度条。 +支持断点续传,无畏大文件。 + + +## 批量检查模型新版本 +你可以按照模型类型,批量检查你的本地模型,在civitai上的新版本。你可以选择多个模型类型。 +![](img/check_model_new_version.jpg) + +检查新版本的时候,每检查完一个模型,都会有一个1秒的延迟,所以速度有点慢。 + +这是为了保护Civitai避免因为本插件而短暂陷入类似DDos的局面。有些云服务商,有类似“免费用户每秒API请求不能超过1次”的保护机制。Civitai还没有这种设置。但我们还是得自觉保护它。因为如果它挂了,对大家都没有好处。 + +**检查完毕之后**,就会如下图,在UI上显示所有找到的新版本的信息。 + +每个模型新版本,都有3个链接。 +* 第一个是这个模型的网页。 +* 第二个是这个新版本的下载地址。 +* 第三个是个按钮,在python端,直接下载新版本到模型目录内。 +这种方式下载,下载详情显示在"Download Model"的区域和命令行窗口中。一次一个任务,不支持多任务。 +![](img/check_model_new_version_output.jpg) + + + +## 根据URL获取模型信息 +如果无法在civitai上找到你的模型的SHA256,但你还是希望能把你的模型连接到一个civitai模型,你可以在本扩展页面,从列表中选择你的模型,并提供一个civitai模型页面的url。 + +点击按钮之后,扩展就会下载那个civitai模型的信息,作为你这个本地模型的信息使用。 + +![](img/get_one_model_info.jpg) + +## 代理 +**如果你是刚更新新版本,你需要重启SD webui再来使用** + +代理输入框在插件页面最下方。 + +**每次填入或清除代理后,都要保存,并用SDwebui设置页面的Reload UI按钮刷新UI** + +然后所有发到civitai的请求就会用代理。 + +有些sock5代理, 需要使用socks5h开头的形式"socks5h://xxxxx"才能生效。 + + + +## 其他设置 +**保存设置按钮, 会保存扫描模型区域,以及其他设置 这两个区域的选项** + +* "一直显示按钮" 是为了方便触屏。 +* "小图模式显示功能按钮" 会开关功能按钮在小图模式的显示 +![](img/other_setting.jpg) + +## 预览图 +Extra Network支持两种预览图命名:`model_name.png` 和 `model_name.preview.png`。其中,`model_name.png`优先级较高。 + +当优先级较高的预览图不存在,他就会自动使用`model_name.preview.png`。 + +这样,你自己创建的预览图 和 网络下载的预览图,能够同时存在,并优先使用你自己创建的。 + +## 关键词 +卡片上,添加关键词按钮,是添加从civitai预览图中得到的关键词,而不是你自己创建的图片的关键词。 + +civitai不是每个图片都有关键词,一个模型中,也不是所有预览图关键词都一样。所以这里是遍历所有civitai预览图信息,加载第一个有关键词的。 + + +## SHA256 +为了创建文件的SHA256,插件需要读取整个文件。对于大尺寸文件,就会很慢。 + +有两种情况,这个SHA256无法从civitai找到对应模型: +* 太老的模型,civitai没有存储SHA256. +* 模型作者,静静的换掉了模型文件,但没有修改描述和版本。所以,虽然网页上看不出来,但实际上civitai上的 和你本地的模型文件,已经不是同一个文件了。 + +这些情况下,你可以在插件上,通过提供模型页面的url,来获取模型信息文件。 + + + +## 新特性 +从v1.5开始,v1.x不再接受任何新特性。所有新特性进入2.x。 + +2.x专注于自定义模型信息,并可能改名为"Model Info Helper"。因为不再是专注Civitai了。 + +从v1.5开始。v1.x进入维护阶段。 + + +Enjoy! + + +## 常见问题 +### 4个卡片按钮不显示 +#### 汉化原因 +下载新版,最新版已经处理汉化导致的问题。**双语汉化插件需要v1.6.1.1之后的版本才开始支持。** + +#### 使用了云端汉化功能 +如果是秋叶启动器,就关闭启动器“云端汉化”功能。如果是专门的云端汉化插件,就换用普通汉化插件。 + +#### 其他情况 +首先,确保你点过了"Refresh Civitai Helper"刷新按钮。 + +然后,如果还有这个问题,那么唯一原因,是你没有使用最新版SD webui。 + +如果你修改过SD webui的文件, 你的更新操作可能会失败。你需要检查git命令行的输出信息,来确定你更新成功了。 + +git在很多时候,会拒绝升级,并告诉你有些冲突需要你手动先解决。如果你不看命令行输出,你就会以为你已经更新成功了,但其实并没有。 + + +### Request model info from civitai +意思就是正在连接civitai,如果没有后面的信息,就是连不上,请挂代理。 + + +### 扫描或获取模型信息失败 +这个插件现在很稳定,所以,这个问题的原因,基本是是因为Civitai拒绝了你的连接请求。 + +Civitai不像那些大网站那么稳定。他网站会挂,会拒绝API连接,还会把API请求转到真人验证页面,来挡住。 + +Civitai还有连接池的设定。基本上,就是同时能允许的最大连接数。一旦达到这个数字,接下来的API连接请求,都会被拒绝。 + +所以,这种时候你只能等一下再试。 + +另外,对于国内用户,还有代理问题。现在国内都要用代理才能连上。 + + +### 扫描之后得到了错误的预览图和模型信息 +坏消息是,有些模型在civitai数据库中,保存的sha256完全是错的。查看下面的issue了解详情: +[https://github.com/civitai/civitai/issues/426](https://github.com/civitai/civitai/issues/426) + +对于这种模型,那这个插件自然就无法获得正确的模型信息和预览图。 + +这种情况下,请删除扫描得到的模型信息和预览图,在插件界面提供正确的模型url来获取。 + +另外,civitai官方有个页面,专门用于回报带有错误sha256的模型: +[https://discord.com/channels/1037799583784370196/1096271712959615100/1096271712959615100](https://discord.com/channels/1037799583784370196/1096271712959615100/1096271712959615100) + +请把这类模型反馈给civitai,好让他们进行修复。 + + + + +### 使用colab时扫描失败 +首先,在google中搜索你看到的错误信息。更有可能是,你碰到的是个colab的问题。 + +然后,如果colab连接了google drive,会有一次性访问文件数量的限制,而导致扫描失败。这是google drive的限制,请自行google搜索了解详情。 + + + diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/README.jp.md b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/README.jp.md new file mode 100644 index 0000000000000000000000000000000000000000..0029c1dad4ccdc3bdba5e582420f8cbecfa9e31e --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/README.jp.md @@ -0,0 +1,236 @@ +# この文書はChatGPTによって英語から翻訳されました。 + +**[寄稿者](https://github.com/zixaphir/Stable-Diffusion-Webui-Civitai-Helper/graphs/contributors)** + +### Language +[中文(ChatGPT)](README.cn.md) +[English](README.md) +[한국어(ChatGPT)](README.kr.md) + +## Civitai Helperについて + +この拡張機能は、Civitaiからモデルとモデルのメタデータをダウンロードできる機能を提供します。Civitaiにホストされているモデルのアクティベーションキーワード、モデルの説明、バージョン情報、モデルのプレビューなどのデータが、安定した拡散WebUIから離れることなく手に入ります。 + +## このバージョンのCivitai Helperについて + +これは私の個人的なバージョンのStable-Diffusion-Webui-Civitai-Helperです。SD-webui v1.5がリリースされたとき、私が使用していたバージョンが壊れたため、完全に新たに始めました。それ以来、自分が必要とする機能を追加し、古いバージョンのsd-webuiとの互換性を維持するために最善の努力をしましたが、古いバージョンは使用しないため、古いバージョンでのテストは行いません。 + +私はできる限りこれを動作させるつもりですが、これは単なる趣味のプロジェクトであり、他のより経験豊富な拡張機能開発者と比べてスキルが圧倒的に不足しています。私は長い間姿を消すことがあり、興味を失った場合は恐らく永遠になるかもしれません。私はエラーを起こしやすく、バグが予想されます。 + +もしもっと興味を持った誰かが開発を引き継ぐことを決定した場合、喜んでこのプロジェクトの制御権を譲渡し、彼らのプロジェクトへの変更を希望される場合にはそれをサポートします。 + +# お知らせ +**この拡張機能は現在、非常に安定しています。もし問題があれば、コンソールログの詳細を確認し、[よくある質問](#よくある質問)を確認してください。** + +# Civitai Helper +この拡張機能は、Civitaiのモデルをより簡単に扱えるようにするためのものです。 + +Civitai: [Civitai Url](https://civitai.com/models/16768/civitai-helper-sd-webui-civitai-extension) + +# 機能 +* 全てのモデルをスキャンし、Civitaiからモデル情報とプレビューをダウンロード +* CivitaiモデルページのURLを使って、ローカルモデルとCivitaiモデル情報を取得 +* CivitaiモデルページのURLから、モデル(情報とプレビューを含む)をSDディレクトリまたはサブディレクトリにダウンロードする。 +* ダウンロードは途中から再開可能 +* ローカルのモデルとCivitai上の新しいバージョンを一括でチェック +* 新しいバージョンのモデルを直接モデルのディレクトリにダウンロード(情報とプレビュー画像を含む) +* 内蔵の**Extra Network**モデルカードを変更し、各カードに以下の機能ボタンを追加しました。 + - 🖼: `replace preview`のテキストをこのアイコンに変更 + - 🌐: このモデルのCivitaiページを新しいタブで開く + - 💡: このモデルのトリガーワードをキーワード入力欄に一括で追加する + - 🏷: このモデルのプレビュー画像で使用されているキーワードを一括で使用する +* 上記の追加機能ボタンは、サムネイルモードにも対応しています。 +* タッチスクリーンデバイス向けに、常に表示されるボタンのオプションを追加しました。 + + +# インストール +SD webui's extensionタブから、`Install from url`のタブに移動。 +このリポジトリのURLをコピーペーストし、インストールする。 + +または、このリポジトリをzipでダウンロードし、`./webui/extensions`へ展開してください。 + +この拡張機能をインストール、またはアップデートするたびに、SD Webui再起動する必要があります。 +この拡張機能は、**UIを再読み込みする**だけでは動作しません。 + +# 使い方 + +## WebUIをアップデート +この拡張機能は`network cards id`を取得する必要があります。この機能は**2023-02-06**に追加されました。 +**SD webuiがこれより前のバージョンである場合は、アップデートする必要があります!** + +## モデルのスキャン +拡張機能タブからCivitai Helperへ。 +Scan modelというボタンがあります。 + +![](img/extension_tab.jpg) + +これをクリックすると、拡張機能がすべてのモデルをスキャンしてSHA256ハッシュを生成し、それを使ってCivitaiからモデル情報とプレビュー画像を取得します。 +**スキャンには時間がかかります。 終了までお待ちください。** + +各モデルに対して、Civitaiからすべてのモデル情報を保存するためのjsonファイルを作成します。このモデル情報ファイルは、modelsディレクトリ内の`Your_model_name.civitai.info`となります。 + +![](img/model_info_file.jpg) + +モデル情報ファイルがすでに存在する場合は、スキップされます。Civitaiでモデルが見つからない場合、空のモデル情報ファイルを作成するので、モデルが2回スキャンされることはありません。 + +### 新しいモデルを追加 +新規のモデルがある場合、もう一度スキャンボタンをクリックするだけで、新しいモデルの情報とプレビューを取得できます。同じモデルを2回スキャンすることはありません。 + +## モデルカード +**(スキャン終了後に使用)** +SD webuiの`Extra Network`タブを開き、モデルカードを表示します。 + +![](img/extra_network.jpg) + + +マウスをモデルカードの下部に移動すると、4つのボタンが表示されます。 + - 🖼: プレビューを置き換えるためのテキストを`replace preview`からこのアイコンに変更します + - 🌐: このモデルのCivitaiページを新しいタブで開きます + - 💡: このモデルのトリガーワードをキーワード入力欄に一括追加します + - 🏷: このモデルのプレビュー画像に使用されているキーワードを一括で使用します + +![](img/model_card.jpg) + +これらのボタンが表示されない場合は、Refresh Civitai Helperをクリックすると、ボタンがカードに再追加されます。 + +![](img/refresh_ch.jpg) + +`Extra Network`が更新されるたびに、余分な変更が削除され、ボタンが消えてしまいます。その場合は、「Refresh Civitai Helper」をクリックして、これらの機能を再度追加する必要があります。 + + +### サムネイル +これらのボタンは、サムネイルをサポートしていますが、SD WebuiのCSSの問題により、現在は常に表示か非表示かのどちらかに制限されています。マウスをスライドして表示することはできません。 +![](img/thumb_mode.jpg) + + +## ダウンロード +**(タスクが一つ完了してから、次のタスクをダウンロードしてください)** +CivitaiモデルページのURLを使用してモデルをダウンロードするには、3つのステップが必要です。 +1. URLを入力し、モデル情報を取得するためにボタンをクリック +2. 拡張機能が自動的にモデル名とタイプを入力します。ダウンロードするサブディレクトリとモデルバージョンを選択 +3. ダウンロードをクリックします +![](img/download_model.jpg) + +ダウンロード状況は、CLIに進行状況バーを表示します。 +断片的に再開することができ、大きなファイルをダウンロードする際にも心配する必要はありません。 + + +## 新しいモデルのバージョンを確認する +モデルの種類に従って、ローカルのモデルを一括でCivitaiの新バージョンがないかをチェックすることができます。複数のモデルの種類を選択できます。 +![](img/check_model_new_version.jpg) + +これを押すと、各モデルをチェックするたびに1秒の遅延が発生するため、速度がやや遅くなります。 + +これは、本拡張機能のユーザーの過失によるDDoSを回避し、Civitaiを保護するために行われます。 +一部のクラウドサービスプロバイダーには、「無料ユーザーのAPIリクエストは1秒あたり1回を超えてはいけない」というような保護があります。Civitaiにはまだこのような設定がありませんが、我々はそれを自衛しなければなりません。 +なぜなら、もしCivitaiがダウンした場合、誰にとっても良いことではないからです。 + +チェックが完了すると、すべての新しいバージョンがUIに表示されます。 + +各モデルの新しいバージョンには、3つのリンクがあります。 +* 最初のものは、このモデルのWebページです。 +* 2つ目は、この新しいバージョンのダウンロードアドレスです。 +* 3つ目は、Python(拡張機能)側で新しいバージョンをモデルディレクトリに直接ダウンロードするボタンです。 +この方法でダウンロードすると、ダウンロードの詳細が「Download Model」の領域とコマンドラインに表示されます。一度に1つのタスクしかサポートされていません。 +![](img/check_model_new_version_output.jpg) + + + +## URLからモデル情報を取得する +Civitai上で自分のモデルのSHA256が見つからない場合でも、自分のモデルをCivitaiモデルに接続したい場合は、この拡張機能のページから、モデルをリストから選択し、CivitaiモデルページのURLを提供することができます。 + +ボタンをクリックすると、拡張機能はCivitaiモデルの情報をダウンロードし、それをローカルモデルの情報として使用します。 + +![](img/get_one_model_info.jpg) + + + +## その他の設定 +**設定保存ボタンを押すと、Scan Modelの設定とその他の設定の両方が保存されます。** + +* Always Display Buttonは、タッチデバイスでの操作を容易にするためです。 +* Show Buttons on Thumb Modeは、小さな画像モードでの機能ボタンの表示を切り替えます。 +![](img/other_setting.jpg) + +## プレビュー +Extra Networkは、2つのプレビュー画像の命名をサポートしています:`model_name.png`と`model_name.preview.png`。 +デフォルトでは自動で`model_name.png`が優先的に使われます。 + +優先度が高いプレビュー画像が存在しない場合は、自動的に`model_name.preview.png`が使用されます。 + +これにより、自分で作成したプレビュー画像とネットからダウンロードしたプレビュー画像を同時に使用し、自分で作成したプレビュー画像を優先的に使用できます。 + +## プロンプト +カード上のUse prompt from preview imageボタンは、Civitaiプレビュー画像から取得したキーワードであり、自分で作成した画像のキーワードではありません。 + +Civitaiにはすべての画像にキーワードがあるわけではなく、1つのモデルに含まれるすべてのプレビュー画像のキーワードが同じであるわけでもありません。したがって、ここではすべてのCivitaiプレビュー画像情報を走査し、最初にキーワードがあるものを読み込みます。 + + +## SHA256 +ファイルのSHA256を作成するために、はファイル全体を読み取る必要があります。大きなファイルの場合、処理が遅くなります。 + +Civitaiで対応するモデルのSHA256が見つからない場合は、次の2つの場合が考えられます: +* 古すぎるモデルには、SHA256が保存されていません。 +* モデルの作成者が静かにモデルファイルを変更しましたが、説明やバージョンを変更していないため、サイト上ではわかりませんが、実際にはCivitaiに保存されているモデルファイルとローカルのモデルファイルは異なるものとなっています。 + +これらの場合は、拡張機能にモデルページのURLを提供することで、モデルの情報ファイルを取得できます。 + +## Feature Request +v1.5以降のv1.xには新機能はありません。すべての新機能は2.xに移行されます。 +2.xでは、カスタムモデル情報にフォーカスし、Civitaiだけではなく、`Model Info Helper`という名称に変更する可能性があります。 +v1.5からv1.xはメンテナンスのフェーズに入ります。 + +お楽しみに! + + +## よくある質問 +### 4つのカードボタンが表示されない +#### ローカライズの問題 +新しいバージョンをダウンロードしてください。 +最新バージョンでは、ローカライズによる問題が解決されています。 +[バイリンガル拡張機能](https://github.com/journey-ad/sd-webui-bilingual-localization)は、v1.6.1.1以降のバージョンでサポートされるようになりました。 + +#### クラウドサービスベースの翻訳機能を使用した +クラウドサービスベースの翻訳機能を使用している場合は、通常のローカライズに変更してください。 + +#### その他の場合 +まず、Refresh Civitai Helperをクリックして更新しましたか? + +それでもこの問題が発生する場合は、おそらく最新バージョンのSD webuiを使用していないためです。 + +SD webuiのファイルを変更した場合、更新操作が失敗する可能性があります。更新が成功したかどうかを確認するには、gitコマンドラインの出力情報を確認する必要があります。 + +gitは、多くの場合、アップグレードを拒否し、手動で解決する必要があるいくつかの競合状態を示します。コマンドライン出力を見ない場合、更新が成功したと思うかもしれませんが、実際には成功していません。 + + +### Request model info from civitai +これはcivitaiに接続しています。情報がない場合は接続できないため、プロキシを使用してください。 + + +### スキャンまたはモデル情報の取得に失敗しました +この拡張機能は現在非常に安定しているため、この問題の原因は基本的にはCivitaiが接続要求を拒否したためです。 + +Civitaiは大きなウェブサイトとは異なり、安定していません。彼らのウェブサイトはダウンしたり、API接続を拒否したり、APIリクエストをCpatchaページに転送してブロックしたりすることがあります。 + +Civitaiには接続プールの上限もあります。基本的に、同時に許可される最大接続数です。この数字に達すると、以降のAPI接続要求はすべて拒否されます。 + +そのため、このような場合はしばらく待ってから再試行するしかありません。 + +### civitaiから誤ったモデル情報とプレビュー画像を取得する(Translated by ChatGPT) +悪いニュースですが、civitaiのデータベースには誤ったsha256で保存されたモデルがいくつかあります。詳細についてはこちらをご覧ください: +[https://github.com/civitai/civitai/issues/426](https://github.com/civitai/civitai/issues/426) + +したがって、これらのモデルについては、この拡張機能では正しいモデル情報やプレビュー画像を取得できません。 + +この場合、モデル情報ファイルを削除し、この拡張機能のタブページでcivitaiのURLから正しいモデル情報を取得する必要があります。 + +また、誤ったsha256を持つこれらのモデルをcivitaiに報告することもできます。 +[https://discord.com/channels/1037799583784370196/1096271712959615100/1096271712959615100](https://discord.com/channels/1037799583784370196/1096271712959615100/1096271712959615100) + +civitaiにそのモデルを報告して修正してもらうようにしてください。 + + +### colabを使用した際にスキャンに失敗する +まず、表示されたエラーメッセージをGoogleで検索してください。おそらくcolabの問題が発生している可能性があります。表示されたエラーメッセージを検索して、原因を特定してください。 + +Google Driveに接続する際には、ファイルへのアクセス数に制限があるため、スキャンが失敗することがよくあります。これはGoogle Drive側の制限です。詳細についてはインターネットで[検索](https://google.com)してください。 diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/README.kr.md b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/README.kr.md new file mode 100644 index 0000000000000000000000000000000000000000..499bdf0c1adf79c723199d727e4a06a6e0e8e9ce --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/README.kr.md @@ -0,0 +1,215 @@ +# 이 문서는 ChatGPT에 의해 영어로 번역되었습니다. + +**[기여자](https://github.com/zixaphir/Stable-Diffusion-Webui-Civitai-Helper/graphs/contributors)** + +## Civitai Helper 소개 + +이 확장 프로그램은 Civitai에서 모델 및 모델 메타데이터를 다운로드하는 기능을 제공합니다. Civitai에서 호스팅되는 모델에 대한 활성화 키워드, 모델 설명, 버전 정보 및 모델 미리보기와 같은 데이터를 안정적인 확산 웹 사용자 인터페이스에서 벗어나지 않고도 손쉽게 얻을 수 있습니다. + +## 이 버전의 Civitai Helper에 대해 + +이것은 Stable-Diffusion-Webui-Civitai-Helper의 개인 버전입니다. SD-webui v1.5가 출시될 때 사용하던 버전이 고장나서 필요한 경우에 완전히 시작했습니다. 그 이후로 원하는 기능을 추가했으며, 이전 버전의 sd-webui와의 호환성을 유지하기 위해 최선의 노력을 기울였지만, 이전 버전을 사용하지 않고 테스트하지 않기 때문에 이전 버전에서의 테스트는 하지 않습니다. + +가능한 한 오랫동안 이 작업을 계속할 계획이지만, 이것은 단순히 취미 프로젝트일 뿐이며, 다른 확장 프로그램 개발자들보다 경험이 풍부하지 않습니다. 때때로 오랜 기간 동안 사라질 가능성이 있으며, 흥미를 잃으면 아마 영원히 그렇게 될 것입니다. 오류를 범하기 쉽고 버그가 예상됩니다. + +더 관심 있는 사람이 개발을 진행하기로 결정하면, 이 프로젝트의 제어권을 기꺼이 양도하고 원하는 경우 그들의 프로젝트로 변경 사항을 밀어넣을 것을 시도할 것입니다. + +# 주의사항 + **이 플러그인은 지금 매우 안정적이며 많은 사용자들이 잘 사용하고 있습니다. 문제가 발생하면, [자주 묻는 질문](#자주-묻는-질문)을 먼저 확인하고 명령 프롬프트 창의 세부 정보를 확인하세요.** + + +# 기능 +* 모든 모델을 스캔하여 Civitai에서 모델 정보 및 미리보기 이미지 다운로드 +* Civitai 모델 페이지 URL을 통해 로컬 모델 및 Civitai 모델 정보 연결 +* Civitai 모델 페이지 URL을 통해 모델(정보 및 미리보기 이미지 포함) 다운로드하여 SD 디렉토리 또는 하위 디렉토리에 저장 +* 이어받기 지원 다운로드 +* 로컬 모델에서 Civitai에 새 버전이 있는지 일괄 확인 +* 새 버전 모델을 SD 모델 디렉토리에 직접 다운로드(정보 및 미리보기 이미지 포함) +* "Extra Network" 모델 카드 내부를 수정하여 다음과 같은 기능 버튼을 추가: + - 🖼: "replace preview" 텍스트를 이 아이콘으로 변경 + - 🌐: 해당 모델의 Civitai 페이지를 새 탭에서 열기 + - 💡: 이 모델의 트리거 단어를 키워드 입력란에 일괄 추가 + - 🏷: 이 모델 미리보기 이미지에 사용된 키워드 사용 +* 위의 추가 기능 버튼은 썸네일 모드를 지원합니다. +* 터치 스크린 사용자를 위해 항상 표시되는 버튼 옵션 추가 + + +# 설치 +이 프로젝트를 zip 파일로 다운로드하고 SD webui 디렉토리/extensions에 압축 해제하면 됩니다. + +이 플러그인을 설치하거나 업그레이드하려면 SD Webui를 완전히 종료하고 다시 시작해야 합니다. UI 다시로드는 작동하지 않습니다. + +# 사용 방법 + +## SD Webui 업데이트 +이 확장 기능은 Extra Network 카드 목록 ID를 가져와야 합니다. 이것은 2023-02-06에 SD Webui에 추가된 것입니다. + +따라서, 만약 사용 중인 버전이 이보다 이전 버전이라면, 먼저 SD Webui를 업데이트해야 합니다! + + +## 모델 스캔 +확장 페이지 "Civitai Helper"로 이동하고 "Scan Model"이라는 버튼을 클릭합니다. + +![](img/extension_tab.jpg) + +클릭하면 모든 모델을 스캔하고 SHA256 코드를 생성하여 Civitai에서 모델 정보 및 미리보기 이미지를 가져옵니다. 스캔에는 시간이 걸리므로 인내심을 가지고 기다려주세요. + +이 확장 기능은 각 모델마다 Civitai에서 얻은 모델 정보를 저장하는 JSON 파일을 생성합니다. 이 파일은 모델이 있는 디렉토리에 "모델 이름.civitai.info"라는 이름으로 저장됩니다. + +![](img/model_info_file.jpg) + +모델 정보 파일이 이미 존재하는 경우 해당 모델은 스캔하지 않습니다. 모델이 Civitai가 아닌 경우 빈 정보 파일이 생성되어 나중에 중복 스캔을 피합니다. + +### 새 모델 추가 +새 모델을 다운로드한 후 스캔 버튼을 다시 클릭하면 됩니다. 이미 스캔된 파일은 다시 스캔하지 않으며 새 모델의 정보와 미리보기 이미지를 자동으로 얻을 수 있습니다. SD Webui를 다시 시작할 필요가 없습니다. + +## 모델 카드 +**(스캔을 완료한 후에 카드 기능을 사용하세요)** +SD Webui의 내장 "Extra Network" 페이지를 열어 모델 카드를 표시합니다. + +![](img/extra_network.jpg) + + +마우스를 모델 카드 아래쪽으로 이동하면 4개의 버튼이 표시됩니다: + - 🖼: "replace preview" 텍스트를이 아이콘으로 변경 + - 🌐:이 모델의 Civitai 페이지를 새 탭에서 엽니다. + - 💡:이 모델의 트리거 단어를 키워드 입력 상자에 추가합니다. + - 🏷:이 모델 미리보기에 사용되는 키워드를 사용합니다. + +![](img/model_card.jpg) + +이러한 추가 버튼이 표시되지 않으면 Refresh Civitai Helper를 클릭하여 다시 추가하십시오. + +![](img/refresh_ch.jpg) + +Extra Network가 새로 고침될 때마다이 추가 수정이 제거되므로 버튼이 사라지면 Refresh Civitai Helper를 클릭하여 기능을 다시 추가해야합니다. + + +### 작은 미리보기 모드 +이러한 기능 버튼은 작은 미리보기 모드를 지원하지만 SD Webui의 CSS 문제로 인해 현재 항상 표시하거나 항상 표시하지 않아야합니다. +![](img/thumb_mode.jpg) + +## 다운로드 +**(한 번에 하나씩, 하나를 다운로드하고 다른 것을 다운로드하세요)** +Civitai 모델 페이지 URL을 통해 모델을 다운로드하려면 3 단계가 필요합니다: +* URL을 입력하고 모델 정보를 가져 오는 버튼을 클릭합니다. +* 확장 프로그램이 모델 이름과 유형을 자동으로 입력합니다. 다운로드 할 하위 디렉토리와 모델 버전을 선택해야합니다. +* 다운로드를 클릭하십시오. +![](img/download_model.jpg) + +다운로드 과정은 진행률 표시 줄이있는 명령 줄 인터페이스에서 표시됩니다. +일시 중지 및 다시 시작을 지원하며 대용량 파일도 문제없이 처리합니다. + + +## 일괄적으로 모델 새 버전 확인 +Civitai에서 새 버전을 확인하기 위해 로컬 모델을 모델 유형 별로 일괄적으로 확인할 수 있습니다. 여러 모델 유형을 선택할 수 있습니다. +![](img/check_model_new_version.jpg) + +새 버전을 확인 할 때마다 모델이 모두 확인 될 때까지 1 초의 지연이 있으므로 속도가 다소 느립니다. + +이것은 Civitai가 이 플러그인으로 인해 일시적으로 DDos와 유사한 상황에 빠지지 않도록 보호하기 위한 것입니다. 일부 클라우드 서비스 제공 업체는 "무료 사용자의 초당 API 요청 수는 1 회를 초과 할 수 없다"는 보호 메커니즘이 있습니다. Civitai는 이러한 설정이 없습니다. 그러나 우리는 여전히 그것을 보호해야합니다. 왜냐하면 그것이 다운되면 모두에게 좋지 않기 때문입니다. + +확인이 완료되면 다음과 같이 UI에 모든 새 버전을 찾은 정보가 표시됩니다. + +각 모델 새 버전에는 3 개의 링크가 있습니다. +* 첫 번째는 이 모델의 웹 페이지입니다. +* 두 번째는이 새 버전의 다운로드 주소입니다. +* 세 번째는 버튼입니다. Python 측에서 새 버전을 모델 디렉토리로 직접 다운로드합니다. +이 방식으로 다운로드하면 "모델 다운로드" 영역과 명령 줄 창에 다운로드 세부 정보가 표시됩니다. 한 번에 하나의 작업만 지원됩니다. +![](img/check_model_new_version_output.jpg) + + + +## URL을 기반으로 모델 정보 가져오기 +Civitai에서 모델의 SHA256을 찾을 수 없지만 여전히 Civitai 모델에 모델을 연결하고 싶다면 해당 확장 프로그램 페이지에서 모델을 선택하고 Civitai 모델 페이지의 URL을 제공할 수 있습니다. + +버튼을 클릭하면 확장 프로그램이 해당 Civitai 모델의 정보를 다운로드하여 로컬 모델의 정보로 사용합니다. + +![](img/get_one_model_info.jpg) + + + +## 기타 설정 +**설정 저장 버튼은 스캔 모델 영역 및 기타 설정 두 영역의 옵션을 저장합니다.** + +* "항상 표시 버튼"은 터치 스크린에서 편리하게 사용하기 위한 것입니다. +* "작은 그림 모드에서 기능 버튼 표시"는 작은 그림 모드에서 기능 버튼을 표시할지 여부를 전환합니다. +![](img/other_setting.jpg) + +## 미리보기 이미지 +Extra Network는 model_name.png 및 model_name.preview.png 두 가지 미리보기 이미지 이름을 지원합니다. 여기서 model_name.png이 우선순위가 높습니다. + +우선순위가 높은 미리보기 이미지가 없으면 자동으로 model_name.preview.png를 사용합니다. + +이렇게 하면 직접 만든 미리보기 이미지와 인터넷에서 다운로드한 미리보기 이미지를 함께 사용할 수 있으며, 우선순위는 직접 만든 이미지가 높습니다. + +## 키워드 +카드에 키워드 추가 버튼은 civitai 미리보기 이미지에서 얻은 키워드를 추가하는 것이며, 사용자가 직접 만든 이미지의 키워드가 아닙니다. + +모든 이미지에 키워드가 있는 것은 아니며, 모델에 따라 미리보기 이미지의 키워드가 모두 같지 않을 수 있습니다. 따라서 여기서는 civitai 모든 미리보기 이미지 정보를 탐색하여 첫 번째 키워드가 있는 이미지를 로드합니다. + + +## SHA256 +파일의 SHA256을 생성하려면 플러그인에서 전체 파일을 읽어야 합니다. 대형 파일의 경우 시스템이 느려질 수 있습니다. + +SHA256은 civitai에서 해당 모델을 찾을 수 없는 두 가지 경우가 있습니다. +* 너무 오래된 모델이므로 civitai에 SHA256이 저장되어 있지 않습니다. +* 모델 작성자가 모델 파일을 조용히 교체했지만 설명 및 버전을 수정하지 않았습니다. 따라서 웹 페이지에서는 확인할 수 없지만 civitai 및 로컬 모델 파일은 이미 다른 파일입니다. + +이러한 경우에는 플러그인에서 모델 페이지 URL을 제공하여 모델 정보 파일을 얻을 수 있습니다 + + +## 자주 묻는 질문 +### 4개의 카드 버튼이 표시되지 않습니다. +#### 한국어 플러그인을 사용했습니다 +새 버전을 다운로드하면, 최신 버전에서 한국어 번역으로 인한 문제가 해결되었습니다. 양방향 다국어 플러그인은 v1.6.1.1 이후 버전부터 지원됩니다. + +#### 클라우드 기반 한국어 플러그인을 사용했습니다 +클라우드 기반 한국어 플러그인을 사용한 경우 일반적인 한국어 플러그인으로 변경하십시오. + + +#### 다른 경우 +먼저 "Refresh Civitai Helper" 버튼을 클릭하여 Civitai Helper를 새로고침했는지 확인하세요. + +그런 다음 이 문제가 계속되는 경우, 유일한 이유는 최신 버전의 SD webui를 사용하지 않았기 때문입니다. + +만약 SD webui의 파일을 수정했다면, 업데이트 작업이 실패할 수 있습니다. 업데이트가 제대로 이루어졌는지 확인하려면 git 명령 줄의 출력 정보를 확인해야 합니다. + +git은 종종 업그레이드를 거부하고, 일부 충돌을 수동으로 해결해야 한다는 메시지를 보여줍니다. 명령 줄 출력을 확인하지 않으면 업그레이드가 성공했다고 잘못 생각할 수 있습니다. + + +### Request model info from civitai +이것은 Civitai에 연결하고 있음을 나타내며, 정보가 없으면 연결할 수 없으므로 프록시를 사용해야 합니다. + + +### 모델 정보 스캔 또는 가져오기 실패 +이 플러그인은 이제 매우 안정적이므로, 이 문제의 원인은 대부분 Civitai가 연결 요청을 거부했기 때문입니다. + +Civitai는 대형 웹사이트와 같이 안정적이지 않습니다. 웹사이트가 다운되거나 API 연결을 거부할 수 있고, API 요청을 실제 검증 페이지로 전환하여 차단할 수도 있습니다. + +Civitai에는 연결 풀 설정이 있습니다. 이는 동시에 허용되는 최대 연결 수입니다. 이 수치에 도달하면 다음 API 연결 요청은 모두 거부됩니다. 이 때는 잠시 기다렸다가 다시 시도해야 합니다. + +또한 국내 사용자들에게는 프록시 문제가 있습니다. 대개는 프록시를 사용해야만 연결할 수 있습니다. + + +### civitai에서 잘못된 모델 정보 및 미리보기 이미지 가져오기 +안타깝게도, civitai의 데이터베이스에 일부 모델이 잘못된 sha256으로 저장되어 있습니다. 자세한 내용은 여기를 확인하십시오: +[https://github.com/civitai/civitai/issues/426](https://github.com/civitai/civitai/issues/426) + +따라서 이 확장 프로그램은 해당 모델의 올바른 모델 정보나 미리보기 이미지를 가져올 수 없습니다. + +이 경우 모델 정보 파일을 제거하고 이 확장 프로그램의 탭 페이지에서 civitai url로 올바른 모델 정보를 가져와야 합니다. + +또한, 잘못된 sha256을 가진 해당 모델을 civitai에 신고할 수 있습니다. +[https://discord.com/channels/1037799583784370196/1096271712959615100/1096271712959615100](https://discord.com/channels/1037799583784370196/1096271712959615100/1096271712959615100) + +civitai에 그 모델을 신고하여 수정할 수 있도록 해주시기 바랍니다. + + +### Colab 사용시 스캔 실패 +먼저 보이는 오류 메시지를 Google에서 검색해보세요. 대개 Colab의 문제일 가능성이 높습니다. + +그리고 Colab이 Google 드라이브에 연결되어 있다면, 파일에 대한 일회성 액세스 제한으로 인해 스캔이 실패할 수 있습니다. 이는 Google 드라이브의 제한 사항으로, 자세한 내용은 Google 검색을 통해 알아보세요. + + + diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/README.md b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/README.md new file mode 100644 index 0000000000000000000000000000000000000000..8de3c66b00cd41811c065727daa5d7b7b6a953de --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/README.md @@ -0,0 +1,252 @@ +**[Contributors](https://github.com/zixaphir/Stable-Diffusion-Webui-Civitai-Helper/graphs/contributors)** + +### Language +[中文(ChatGPT)](README.cn.md) +[日本語(ChatGPT)](README.jp.md) +[한국어(ChatGPT)](README.kr.md) + +## About Civitai Helper +This extension provides the ability to download models and model metadata from Civitai. Data such as activation keywords, model description, version information, and model previews for models hosted on Civitai can be at your fingertips without having to navigate away from stable diffusion webui. + +## About This Version of Civitai Helper +This is my personal version of Stable-Diffusion-Webui-Civitai-Helper. I started it entirely because the version I was using broke when SD-webui v1.5 came out and I needed it to work. Since then, I have added functionality that I wanted and have made best-effort attempts to maintain compatibility with older versions of sd-webui, but I do not use older versions and therefore do not test on older versions. + +I intend to keep this working for as long as I am able, but this is simply a hobby project and I am nowhere near as skilled as other extension developers with more experience. I am likely to dip out for long periods at a time, possibly forever if I lose interest. I am prone to errors and bugs are to be expected. + +If a more interested part decides to pick up development, I will happily cede control of this project and attempt to push my changes to their projects if they are wanted. + +# Civitai Helper +Stable Diffusion Webui Extension for Civitai, to handle your models much more easily. + +Civitai: ![Civitai Url](https://civitai.com/models/16768/civitai-helper-sd-webui-civitai-extension) + +# Features +* Scans all models to download model information and preview images from Civitai. +* Link local model to a civitai model by civitai model's url +* Download a model(with info+preview) by Civitai Url into SD's model folder or subfolder. +* Downloading can resume at break-point, which is good for large file. +* Checking all your local model's new version from Civitai +* Download a new version directly into SD model folder (with info+preview) +* Modified Built-in "Extra Network" cards, to add the following buttons on each card: + - 🖼️: Modified "replace preview" text into this icon + - 🌐: Open this model's Civitai url in a new tab + - 💡: Add this model's trigger words to prompt + - 🏷️: Use this model's preview image's prompt + - ✏️: Rename model + - ❌: Remove/Delete model +* Above buttons support thumbnail mode of Extra Network in versions of webui prior to 1.5.0. + * Thumbnail mode was removed in v1.5.0 of webui and replaced customizable model card sizes. +* Option to always show additional buttons, to work with touchscreen. +* To the best of my knowledge, this extension should still work in versions of webui prior to v1.5.0, but it is not tested. I make best-effort attempts to write code that *should* maintain compatibility with older versions, but if you have run into problems, please file an issue and I'll attempt to resolve it. + +# Install +Go to SD webui's extension tab, go to `Install from url` sub-tab. +Copy this project's url into it, click install. + +Alternatively, download this project as a zip file, and unzip it to `Your SD webui folder/extensions`. + +Everytime you install or update this extension, you need to shutdown SD Webui and Relaunch it. Just "Reload UI" won't work for this extension. + +Done. + +## Branches +Development of this extension happens in three development branches: +* **`master`**: The current version of the extension intended for end users. Out-of-version changes to this branch only exist to hotfix critical issues found after the release of a new version. +* **`dev`**: The active development version of this extension. This will always have the most up-to-date changes but is also the most likely to contain bugs +* **`v1.6ONLY`**: Not supported and not really intended for anybody except for me. Slowest to update and less tested than other branches, this branch only runs on the latest version of webui. Compatibility code for older versions is actively removed, and whether I'm running it on webui stable or webui dev is in flux. In theory, this is the most optimized version of the extension, but it's often just me chasing waterfalls. Do not submit issues if you use this branch. They will be marked as invalid, closed, and otherwise ignored. + +## Update Your SD Webui +This extension need to get extra network's cards id. Which is added since **2023-02-06**. +**If your SD webui is an earlier version, you need to update it!** + +### Some of the following information may not up-to-date. Most functionality should be the same or similar, but many changes post-v1.6 have not been documented as of yet. Images may not match 1:1 with the current state of the extension. + +## Scanning Models +Go to extension tab "Civitai Helper". There is a button called "Scan model". + +![](img/extension_tab.jpg) + +Click it and the extension will scan all your models to generate SHA256 hashes, using them to retreive model information and preview images from Civitai. + +**Scanning takes time, just wait it finish** + +For each model, it will create two files to save all model info from Civitai. These model info files will be `[model_name].civitai.info` and `[model_name].json` in your model folder. + +![](img/model_info_file.jpg) + +If a model info file already exists, by default it will be skipped. If a model cannot be found in Civitai, a minimal model info file will be created with any information that can be extracted from the model. By default, a model with model pre-existing model info files will not be scanned. + +### Adding New Models +When you have some new models, just click scan button again to get new model's information and preview images. Only new models will be scanned with default options. + +## Model Card + +### The following section is outdated! +The following text block and image only applies to Stable Diffution Webui versions before v1.5.0. While the added buttons are still up-to-date, the "Extra Networks" tab button has been removed and is now always active by default. + +--- + +**(Use this only after scanning finished)** +Open SD webui's build-in "Extra Network" tab, to show model cards. + +![](img/extra_network.jpg) + + +Move your mouse on to the bottom of a model card. It will show 4 icon buttons: + - 🖼: Replace preview (a build-in button, modified from text to icon) + - 🌐: Open this model's Civitai url in a new tab + - 💡: Add this model's trigger words to prompt + - 🏷: Use this model's preview image's prompt + - ✏️: Rename model + - ❌: Remove/Delete model + +![](img/model_card.jpg) + +## Webui Metadata Editor +As of v1.7.0, this extension also downloads data for Webui's Metadata Editor by default. This data includes information you'd previously have to read unruly JSON files or navigate to Civitai to read. + +![](img/webui_metadata_editor.png) + +This data can be accessed by clicking the metadata button on the model card. + +![](img/webui_metadata_button.png) + +## Download +To download a model by Civitai Model Page's Url, you need 3 steps: +* Fill the Civitai URL or Model ID +* Click "1. Get Model Information by Civitai Url. +* It will fill model name, type, sub-folder, and model version automatically, but you can change the sub-folder and model version if you need to. + * If you need to add more sub-folders, you must do this by navigating to the model directory on the system running your webui version. +* Click download. + +![](img/download_model.jpg) + +Detail will be displayed on console log, with a progress bar. +Downloading can resume from break-point, so no fear for large file. + +## Checking Model's New Version +You can checking your local model's new version from civitai by model types. You can select multiple model types. + +![](img/check_model_new_version.jpg) + +The checking process has a small delay after each model's new version checking request. So it is a little slow. + +This is to protect Civitai from issue like DDos from this extension. There is no good for us if it is down. + +**After checking process done**, it will display all new version's information on UI. + +There are 3 urls for each new version. +* First one is model's civitai page. +* Second one is new version's download url. +* Third one is a button to download it into your SD's model folder with python. +With this one, output information is on "Download Model" section's log and console log. **One task at a time**. + +![](img/check_model_new_version_output.png) + +## Get Model Info By Url +This is used to force a local model links to a Civitai model. For example, you converted a model's format or pruned it. Then it can not be found on civitai when scanning. + +In that case, if you still want to link it to a civitai model. You can use this funcion. + +Choose this model from list, then offer a civitai model page's url. + +After clicking button, extension will download that civitai model's info and preview image for the local file you picked. + +![](img/get_one_model_info.jpg) + +## Proxy +**If you are updating to new version, you need to re-lanuch SD webui before using it.** + +Proxy textbox is at the bottom of extension tab. + +**Each time you fill or clear a proxy value, you need to save setting, and Re-load UI with setting tab's reload button.** + +Then all requests to civitai will use the proxy. + +For some sock5 proxy, need to be used as "socks5h://xxxxx". + +## Other Setting +**The Save Setting button, will save both "Scan Model"'s setting and other setting.** + +* "Always Display Button" is good for touch screen. +* "Show Buttons on Thumb Mode" will turn on/off additional Buttons on thumbnail. + * Thumbnail Mode was removed in v1.5.0 of webui. + +![](img/other_setting.jpg) + +## Preview Image +Extra network uses both `model_file.png` and `model_file.preview.png` as preview image. But `model_file.png` has higher priority, because it is created by yourself. + +When you don't have the higher priority one, it will use the other automatically. + +## Prompt +When you click the button "Use prompt from preview image", it does not use the prompt from your own preview image. It uses the one from civitai's preview image. + +On civitai, a model's preview images may not has prompt. This extension will check this model's all civitai preview images' information and use the first one has prompt in it. + +## SHA256 +To create a file SHA256, it need to read the whole file to generate a hash code. It gonna be slow for large files. + +Also, extension uses Memory Optimized SHA256, which won't stuck your system and works with colab. + +There are 2 cases this hash code can not find the model on civitai: +* Some old models, which do not have SHA256 code on civitai. +* The model's owner changed file on civitai, but does not change version name and description. So, the file on civitai is actually not the one on your manchine. + +In these cases, you can always link a model to civitai by filling its URL in this extension. + +## Feature Request +Feel free to submit feature requests, but pull requests are preferred. + +Enjoy! + +## Pull Requests +All pull requests should target the dev branch. For those who take a stab at the code, I apologize for the lack of consistency in coding style, naming, and other syntactical oddities. At some point, I intend to clean up the code and have everything pass linting, but we're not there yet. + +## Common Issue +### 4 Buttons on card didn't show +#### Localization +There was a Localization issue if you are not using English version of SD webui. This is fixed in the latest version of this extension. **Bilingual localization extension is supported by PR since v1.6.1.1.** + +##### Using cloud based localization extension +Turn off cloud based localization extension, use normal localization extension. + +#### Other case +First of all, make sure you clicked "Refresh Civitai Helper" button. + +If issue is still there, then only reason is you are not using the latest SD webui. So, Make sure you updated it. + +Your update could be failed if you have modified SD webui's file. You need to check git command's console log to make sure it is updated. + +In many cases, git will just refuse to update and tell you there are some conflicts need you to handle manually. If you don't check the consloe log, you will think your SD webui is updated, but it is not. + +### Request, Scan or Get model info failed +Usually the reason for this most likely is the connection to Civitai API service failed. This can be for a number of reasons. + +Sometimes Civitai can be down or refuse your API connection. Civitai has a connection pool setting. Basicly, it's a max connection number that civitai can have at the same time. So if there are already too manny connections on civitai, it will refuse your API connection. + +In those cases, the only thing you can do is just wait a while then try again. I suggest making a cup of tea! + +### Get Wrong model info and preview images from civitai +A bad news is, some models are saved with a wrong sha256 in civitai's database. Check here for more detail: +[https://github.com/civitai/civitai/issues/426](https://github.com/civitai/civitai/issues/426) + +So, for those models, this extension can not get the right model info or preview images. + +In this case, you have to remove the model info file and get the right model info by a civitai url on this extension's tab page. + +Also, you can report those models with wrong sha256 to civitai at following page: +[https://discord.com/channels/1037799583784370196/1096271712959615100/1096271712959615100](https://discord.com/channels/1037799583784370196/1096271712959615100/1096271712959615100) + +Please report that model to civitai, so they can fix it. + +### Scanning fail when using colab +First of, search your error message with google. Most likely, it will be a colab issue. + +If you are sure it is a out of memory issue when scanning models, and you are using this extension's latest version, then there is nothing we can do. + +Since v1.5.5, we've already optimized the SHA256 function to the top. So the only 2 choices for you are: +* try again +* or use a pro account of colab. + +### [Changes](https://github.com/zixaphir/Stable-Diffusion-Webui-Civitai-Helper/blob/master/README.md) diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/icon/.keep b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/icon/.keep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/all_in_one.png b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/all_in_one.png new file mode 100644 index 0000000000000000000000000000000000000000..7db1d22b4c770224d6619813cdf975fce52eb21e --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/all_in_one.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bedc58968db1a83c84e57d94327c4e27f5cd6fbd93c45a59627a5257b06c8a47 +size 1104345 diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/all_the_mothers.png b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/all_the_mothers.png new file mode 100644 index 0000000000000000000000000000000000000000..077e808d4bffa77c927f60f4e26b025f51290553 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/all_the_mothers.png differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/check_model_new_version.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/check_model_new_version.jpg new file mode 100644 index 0000000000000000000000000000000000000000..1d05e4490b2ff7a3a82edee277a7130aee149d55 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/check_model_new_version.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/check_model_new_version_output.png b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/check_model_new_version_output.png new file mode 100644 index 0000000000000000000000000000000000000000..11fa31d7ecba1679beba9d6cde5a2bb8c02269c4 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/check_model_new_version_output.png differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/download_model.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/download_model.jpg new file mode 100644 index 0000000000000000000000000000000000000000..f6c052ccfca850008f7a87e7dfd8b45d2ec17209 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/download_model.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/extension_tab.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/extension_tab.jpg new file mode 100644 index 0000000000000000000000000000000000000000..a9e0dbe1d1164117cd11064ebbf220377a495c2e Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/extension_tab.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/extra_network.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/extra_network.jpg new file mode 100644 index 0000000000000000000000000000000000000000..25b2acbdb4afefb348f058a9d45fd8a090eda8ca Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/extra_network.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/get_one_model_info.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/get_one_model_info.jpg new file mode 100644 index 0000000000000000000000000000000000000000..9de82e031c5320d45fef054026ada12f3026e776 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/get_one_model_info.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/model_card.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/model_card.jpg new file mode 100644 index 0000000000000000000000000000000000000000..6ceb33f68f91f3bf3eef625bc18ee427e1002456 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/model_card.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/model_info_file.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/model_info_file.jpg new file mode 100644 index 0000000000000000000000000000000000000000..d9e36c8e69a3feb90034a2c205a4d4c27aa43bd7 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/model_info_file.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/other_setting.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/other_setting.jpg new file mode 100644 index 0000000000000000000000000000000000000000..9b9a342410e65bcf616978530c79a636ab173f4e Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/other_setting.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/refresh_ch.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/refresh_ch.jpg new file mode 100644 index 0000000000000000000000000000000000000000..93b4ab70f94ff1718cebc3880f34c4dce45ddfc8 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/refresh_ch.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/thumb_mode.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/thumb_mode.jpg new file mode 100644 index 0000000000000000000000000000000000000000..1044e68cf89c9f0faa1e1f74a8a7cbb5d97809d6 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/thumb_mode.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/blame_sdweui_update_to_this_ext.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/blame_sdweui_update_to_this_ext.jpg new file mode 100644 index 0000000000000000000000000000000000000000..59ccb37188850686ef48248d2f565098403a0d99 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/blame_sdweui_update_to_this_ext.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/changed_model_folder_name_then_forget_part1.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/changed_model_folder_name_then_forget_part1.jpg new file mode 100644 index 0000000000000000000000000000000000000000..76acd20220c8f3c22619e5d5f4541ce2aeefbeb4 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/changed_model_folder_name_then_forget_part1.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/changed_model_folder_name_then_forget_part2.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/changed_model_folder_name_then_forget_part2.jpg new file mode 100644 index 0000000000000000000000000000000000000000..0f231f6f8b9fb981fd9b52ff04aa44f26abf4ddb Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/changed_model_folder_name_then_forget_part2.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/changed_model_folder_name_then_forget_part3.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/changed_model_folder_name_then_forget_part3.jpg new file mode 100644 index 0000000000000000000000000000000000000000..fc032c496932463b958444048200dd43068984ca Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/changed_model_folder_name_then_forget_part3.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/changed_model_folder_name_then_forget_part4.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/changed_model_folder_name_then_forget_part4.jpg new file mode 100644 index 0000000000000000000000000000000000000000..6ffd6197fe4889e946dd2c24d2192b8940c1bddc Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/changed_model_folder_name_then_forget_part4.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/css_issue_part1.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/css_issue_part1.jpg new file mode 100644 index 0000000000000000000000000000000000000000..26567730c1a3253244d9a8230b90456add779b69 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/css_issue_part1.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/css_issue_part2.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/css_issue_part2.jpg new file mode 100644 index 0000000000000000000000000000000000000000..54b9a2cd28a2f57e43a3f2f0b92c68dbd1c52123 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/css_issue_part2.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/css_issue_part3.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/css_issue_part3.jpg new file mode 100644 index 0000000000000000000000000000000000000000..8c0d4ce269af46779bdf4981a7bebf40a4768559 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/css_issue_part3.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/css_issue_part4.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/css_issue_part4.jpg new file mode 100644 index 0000000000000000000000000000000000000000..9ddad8e72b21e5148b14a1e4170a6a774a8d2c4c Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/css_issue_part4.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/did_not_relaunch_sdwebui.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/did_not_relaunch_sdwebui.jpg new file mode 100644 index 0000000000000000000000000000000000000000..984228d4cffa6367e5191bb756350f7611b42091 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/did_not_relaunch_sdwebui.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/do_not_even_use_this_ext.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/do_not_even_use_this_ext.jpg new file mode 100644 index 0000000000000000000000000000000000000000..e4be133af1c95e21fce1e6b9b4a26f312722ccb5 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/do_not_even_use_this_ext.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/have_not_scan_model.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/have_not_scan_model.jpg new file mode 100644 index 0000000000000000000000000000000000000000..fc20a79fd6c096d1a2df8d612d06f89a323cdbbd Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/have_not_scan_model.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/have_not_update_sdwebui.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/have_not_update_sdwebui.jpg new file mode 100644 index 0000000000000000000000000000000000000000..3fb704c1f4ce3ef9ea423d2bab3eeff9eab7fe4f Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/have_not_update_sdwebui.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/request_a_feature_it_already_has.jpg b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/request_a_feature_it_already_has.jpg new file mode 100644 index 0000000000000000000000000000000000000000..2e1eab1df869661eab9475344d28eed4c59129e0 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/user_claim_wall/request_a_feature_it_already_has.jpg differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/webui_metadata_button.png b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/webui_metadata_button.png new file mode 100644 index 0000000000000000000000000000000000000000..2133e87ef4dfa93b5b41a4eb756bd83d3bd8adb6 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/webui_metadata_button.png differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/webui_metadata_editor.png b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/webui_metadata_editor.png new file mode 100644 index 0000000000000000000000000000000000000000..fce4b27f23aaea4315abb2932de68043db070f8b Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/img/webui_metadata_editor.png differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/javascript/civitai_helper.js b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/javascript/civitai_helper.js new file mode 100644 index 0000000000000000000000000000000000000000..cea679c53bb3160c08de4a0ee9704521e50782d0 --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/javascript/civitai_helper.js @@ -0,0 +1,1012 @@ +"use strict"; + + +function ch_convert_file_path_to_url(path) { + let prefix = "file="; + let path_to_url = path.replaceAll('\\', '/'); + return prefix+path_to_url; +} + +function ch_img_node_str(path) { + return ``; +} + +function ch_gradio_version() { + let foot = gradioApp().getElementById("footer"); + if (!foot) { return null; } + + let versions = foot.querySelector(".versions"); + if (!versions) { return null; } + + if (versions.textContent.indexOf("gradio: 3.16.2")>0) { + return "3.16.2"; + } else { + return "3.23.0"; + } +} + + +// send msg to python side by filling a hidden text box +// then will click a button to trigger an action +// msg is an object, not a string, will be stringify in this function +function send_ch_py_msg(msg) { + console.log("run send_ch_py_msg"); + let js_msg_txtbox = gradioApp().querySelector("#ch_js_msg_txtbox textarea"); + if (js_msg_txtbox && msg) { + // fill to msg box + js_msg_txtbox.value = JSON.stringify(msg); + js_msg_txtbox.dispatchEvent(new Event("input")); + } + +} + +// get msg from python side from a hidden textbox +// normally this is an old msg, need to wait for a new msg +function get_ch_py_msg() { + console.log("run get_ch_py_msg"); + const py_msg_txtbox = gradioApp().querySelector("#ch_py_msg_txtbox textarea"); + if (py_msg_txtbox && py_msg_txtbox.value) { + console.log("find py_msg_txtbox"); + console.log("py_msg_txtbox value: "); + console.log(py_msg_txtbox.value); + return py_msg_txtbox.value; + } else { + return ""; + } +} + + +// get msg from python side from a hidden textbox +// it will try once in every sencond, until it reach the max try times +const get_new_ch_py_msg = (max_count=5) => new Promise((resolve, reject) => { + console.log("run get_new_ch_py_msg"); + + let count = 0; + let new_msg = ""; + let find_msg = false; + const interval = setInterval(() => { + const py_msg_txtbox = gradioApp().querySelector("#ch_py_msg_txtbox textarea"); + count++; + + if (py_msg_txtbox && py_msg_txtbox.value) { + console.log("find py_msg_txtbox"); + console.log("py_msg_txtbox value: "); + console.log(py_msg_txtbox.value); + + new_msg = py_msg_txtbox.value; + if (new_msg != "") { + find_msg = true; + } + } + + if (find_msg) { + //clear msg in both sides + py_msg_txtbox.value = ""; + py_msg_txtbox.dispatchEvent(new Event("input")); + + resolve(new_msg); + clearInterval(interval); + } else if (count > max_count) { + //clear msg in both sides + py_msg_txtbox.value = ""; + py_msg_txtbox.dispatchEvent(new Event("input")); + + reject(''); + clearInterval(interval); + } + + }, 1000); +}); + + +function getActiveTabType() { + const currentTab = get_uiCurrentTabContent(); + switch (currentTab.id) { + case "tab_txt2img": + return "txt2img"; + case "tab_img2img": + return "img2img"; + } + return null; +} + +function getExtraTabs(prefix) { + return gradioApp().getElementById(prefix + "_extra_tabs"); +} + +function getActivePrompt() { + const currentTab = get_uiCurrentTabContent(); + switch (currentTab.id) { + case "tab_txt2img": + return currentTab.querySelector("#txt2img_prompt textarea"); + case "tab_img2img": + return currentTab.querySelector("#img2img_prompt textarea"); + } + return null; +} + +function getActiveNegativePrompt() { + const currentTab = get_uiCurrentTabContent(); + switch (currentTab.id) { + case "tab_txt2img": + return currentTab.querySelector("#txt2img_neg_prompt textarea"); + case "tab_img2img": + return currentTab.querySelector("#img2img_neg_prompt textarea"); + } + return null; +} + + +//button's click function +async function open_model_url(event, model_type, search_term) { + console.log("start open_model_url"); + + //get hidden components of extension + let js_open_url_btn = gradioApp().getElementById("ch_js_open_url_btn"); + if (!js_open_url_btn) { + console.log("Failed to find js_open_url_btn"); + return; + } + + //msg to python side + let msg = { + "action": "", + "model_type": "", + "search_term": "", + "prompt": "", + "neg_prompt": "", + }; + + msg["action"] = "open_url"; + msg["model_type"] = model_type; + msg["search_term"] = search_term; + msg["prompt"] = ""; + msg["neg_prompt"] = ""; + + // fill to msg box + send_ch_py_msg(msg); + + //click hidden button + js_open_url_btn.click(); + + // stop parent event + event.stopPropagation(); + event.preventDefault(); + + //check response msg from python + let new_py_msg = null; + try { + new_py_msg = await get_new_ch_py_msg(); + } catch (error) { + console.log(error); + } + + console.log("new_py_msg:"); + console.log(new_py_msg); + + //check msg + if (new_py_msg) { + let py_msg_json = JSON.parse(new_py_msg); + //check for url + if (py_msg_json && py_msg_json.content) { + if (py_msg_json.content.url) { + window.open(py_msg_json.content.url, "_blank"); + } + } + } + + console.log("end open_model_url"); + +} + +function add_trigger_words(event, model_type, search_term) { + console.log("start add_trigger_words"); + + //get hidden components of extension + let js_add_trigger_words_btn = gradioApp().getElementById("ch_js_add_trigger_words_btn"); + if (!js_add_trigger_words_btn) { + return; + } + + + //msg to python side + let msg = { + "action": "", + "model_type": "", + "search_term": "", + "prompt": "", + "neg_prompt": "", + }; + + msg["action"] = "add_trigger_words"; + msg["model_type"] = model_type; + msg["search_term"] = search_term; + msg["neg_prompt"] = ""; + + // get active prompt + let act_prompt = getActivePrompt(); + msg["prompt"] = act_prompt.value; + + // fill to msg box + send_ch_py_msg(msg); + + //click hidden button + js_add_trigger_words_btn.click(); + + console.log("end add_trigger_words"); + + event.stopPropagation(); + event.preventDefault(); + + +} + +function use_preview_prompt(event, model_type, search_term) { + console.log("start use_preview_prompt"); + + //get hidden components of extension + let js_use_preview_prompt_btn = gradioApp().getElementById("ch_js_use_preview_prompt_btn"); + if (!js_use_preview_prompt_btn) { + return; + } + + //msg to python side + let msg = { + "action": "", + "model_type": "", + "search_term": "", + "prompt": "", + "neg_prompt": "", + }; + + msg["action"] = "use_preview_prompt"; + msg["model_type"] = model_type; + msg["search_term"] = search_term; + + // get active prompt + let act_prompt = getActivePrompt(); + msg["prompt"] = act_prompt.value; + + // get active neg prompt + let neg_prompt = getActiveNegativePrompt(); + msg["neg_prompt"] = neg_prompt.value; + + // fill to msg box + send_ch_py_msg(msg); + + //click hidden button + js_use_preview_prompt_btn.click(); + + console.log("end use_preview_prompt"); + + event.stopPropagation(); + event.preventDefault(); + +} + + +async function remove_card(event, model_type, search_term) { + console.log("start remove_card"); + + //get hidden components of extension + let js_remove_card_btn = gradioApp().getElementById("ch_js_remove_card_btn"); + if (!js_remove_card_btn) { + return; + } + + // must confirm before removing + let rm_confirm = "\nConfirm to remove this model and all related files. This process is irreversible."; + if (!confirm(rm_confirm)) { + return; + } + + //msg to python side + let msg = { + "action": "", + "model_type": "", + "search_term": "", + } + + msg["action"] = "remove_card"; + msg["model_type"] = model_type; + msg["search_term"] = search_term; + + // fill to msg box + send_ch_py_msg(msg) + + //click hidden button + js_remove_card_btn.click(); + + // stop parent event + event.stopPropagation() + event.preventDefault() + + //check response msg from python + let new_py_msg = ""; + try { + new_py_msg = await get_new_ch_py_msg(); + } catch (error) { + console.log(error); + new_py_msg = error; + } + + console.log("new_py_msg:"); + console.log(new_py_msg); + + //check msg + let result = "Done"; + //check msg + if (new_py_msg) { + result = new_py_msg; + } + + if (result == "Done") { + refresh_cards_list(); + } + + console.log("end remove_card"); + +} + + +async function rename_card(event, model_type, search_term, model_name) { + console.log("start rename_card"); + + //get hidden components of extension + let js_rename_card_btn = gradioApp().getElementById("ch_js_rename_card_btn"); + if (!js_rename_card_btn) { + return; + } + + // must confirm before removing + let rename_prompt = "\nRename this model to:"; + let new_name = prompt(rename_prompt, model_name); + if (!new_name) { + return; + } + + //msg to python side + let msg = { + "action": "", + "model_type": "", + "search_term": "", + "new_name": "", + } + + msg["action"] = "rename_card"; + msg["model_type"] = model_type; + msg["search_term"] = search_term; + msg["new_name"] = new_name; + + // fill to msg box + send_ch_py_msg(msg) + + //click hidden button + js_rename_card_btn.click(); + + // stop parent event + event.stopPropagation() + event.preventDefault() + + //check response msg from python + let new_py_msg = ""; + try { + new_py_msg = await get_new_ch_py_msg(); + } catch (error) { + console.log(error); + new_py_msg = error; + } + + console.log("new_py_msg:"); + console.log(new_py_msg); + + //check msg + let result = "Done"; + //check msg + if (new_py_msg) { + result = new_py_msg; + } + + if (result == "Done") { + refresh_cards_list(); + } + + console.log("end rename_card"); + +} + + +function replace_preview(e, page, type, name) { + // we have to create a whole hidden editor window to access preview replace functionality + extraNetworksEditUserMetadata(e, page, type, name); + + // the editor window takes quite some time to populate + waitForEditor(page, type, name).then(editor => { + // Gather the buttons we need to both replace the preview and close the editor + let cancel_button = editor.querySelector('.edit-user-metadata-buttons button:first-of-type'); + let replace_preview_button = editor.querySelector('.edit-user-metadata-buttons button:nth-of-type(2)'); + + replace_preview_button.click(); + cancel_button.click(); + }); +} + + +// download model's new version into SD at python side +function ch_dl_model_new_version(event, model_path, version_id, download_url, model_type) { + console.log("start ch_dl_model_new_version"); + + // must confirm before downloading + let dl_confirm = "\nConfirm to download.\n\nCheck Download Model Section's log and console log for detail."; + if (!confirm(dl_confirm)) { + return; + } + + //get hidden components of extension + let js_dl_model_new_version_btn = gradioApp().getElementById("ch_js_dl_model_new_version_btn"); + if (!js_dl_model_new_version_btn) { + return; + } + + //msg to python side + let msg = { + "action": "", + "model_path": "", + "version_id": "", + "download_url": "", + }; + + msg["action"] = "dl_model_new_version"; + msg["model_path"] = model_path; + msg["version_id"] = version_id; + msg["download_url"] = download_url; + msg["model_type"] = model_type; + + // fill to msg box + send_ch_py_msg(msg); + + //click hidden button + js_dl_model_new_version_btn.click(); + + console.log("end dl_model_new_version"); + + event.stopPropagation(); + event.preventDefault(); + + +} + + +function refresh_cards_list() { + console.log("refresh card list"); + //refresh card list + let active_tab = getActiveTabType(); + console.log(`get active tab id: ${active_tab}`); + if (active_tab) { + let refresh_btn_id = `${active_tab}_extra_refresh`; + let refresh_btn = gradioApp().getElementById(refresh_btn_id); + if (refresh_btn) { + console.log(`click button: ${refresh_btn_id}`); + refresh_btn.click(); + } + } +} + +function processCards(tab, extra_tab_els) { + if (!(opts && "ch_always_display" in opts)) { + // Lobe theme can cause a race condition. + console.log("Waiting for webui settings to become available"); + console.log(opts); + const try_again = function () { + processCards(tab, extra_tab_els); + } + return setTimeout(try_again, 500); + } + + const prefix_length = tab.length + 1; + for (const el of extra_tab_els) { + const model_type = el.id.slice(prefix_length, -6); + const cards = el.querySelectorAll('.card'); + for (const card of cards) { + processSingleCard(tab, getShortModelTypeFromFull(model_type), card); + } + } +} + + +function getModelCardsEl(prefix, model_type) { + const id = prefix + "_" + model_type + "_cards"; + return gradioApp().getElementById(id); +} + + +function waitForExtraTabs(tab, extra_tabs) { + function findTabs() { + const tab_elements = []; + for (const extra_tab of extra_tabs) { + const extra_tab_el = getModelCardsEl(tab, extra_tab); + + if (extra_tab_el == null) { + + // XXX lycoris models do not have their own tab in sdwebui 1.5 + // most of the time. In the case that there is a LyCoris tab, + // it would have been added at the same time as the others, + // making it almost impossible to be null by the time we're at + // this point in the code if the other tabs are loaded. + if (extra_tab == 'lycoris') { continue; } + + return null; + } + + tab_elements.push(extra_tab_el); + } + return tab_elements; + } + + const tab_elements = findTabs(tab, extra_tabs); + if (tab_elements) { + processCards(tab, tab_elements); + } + + const observer = new MutationObserver(records => { + let tab_elements; + for (const record of records) { + if (record.type != "childList") { + continue; + } + + tab_elements = findTabs(tab, extra_tabs); + if (!tab_elements) { + return; + } + + processCards(tab, tab_elements); + return; + } + }); + + const extra_networks = getExtraTabs(tab); + + const options = { + subtree: true, + childList: true, + }; + + observer.observe(extra_networks, options); + +} + + +function waitForEditor(page, type, name) { + const id = page + '_' + type + '_edit_user_metadata'; + + return new Promise(resolve => { + let name_field; + const gradio = gradioApp(); + + const editor = gradio.getElementById(id); + const popup = gradio.querySelector(".global-popup"); + + if (popup != null) { + // hide the editor window so it doesn't get in the user's + // way while we wait for the replace preview functionality + // to become available. + popup.style.display = "none"; + } + + // not only do we need to wait for the editor, + // but also for it to populate with the model metadata. + if (editor != null) { + name_field = editor.querySelector('.extra-network-name'); + if (name_field.textContent.trim() == name) { + return resolve(editor); + } + } + + const observer = new MutationObserver(() => { + const editor = gradioApp().getElementById(id); + let name_field; + if (editor != null) { + name_field = editor.querySelector('.extra-network-name'); + if (name_field.textContent.trim() == name) { + resolve(editor); + observer.disconnect(); + } + } + }); + + observer.observe(document.body, { + subtree: true, + childList: true, + }); + }); +} + + +function getShortModelTypeFromFull(model_type_full) { + switch (model_type_full) { + case "textual_inversion": + return "ti"; + case "hypernetworks": + return "hyper"; + case "checkpoints": + return "ckp"; + case "lora": + case "lycoris": + return model_type_full; + } +} + + +function getLongModelTypeFromShort(model_type_short) { + switch (model_type_short) { + case "ti": + return "textual_inversion"; + case "hyper": + return "hypernetworks"; + case "ckp": + return "checkpoints"; + case "lora": + case "lycoris": + return model_type_short; + } +} + + +function isThumbMode(extra_network_node) { + if (extra_network_node?.className == "extra-network-thumbs") { + return true; + } + return false; +} + + +function processSingleCard(active_tab_type, active_extra_tab_type, card) { + let metadata_button = null; + let additional_node = null; + let replace_preview_btn = null; + let ul_node = null; + let search_term_node = null; + let model_name = ""; + let search_term = ""; + let model_type = active_extra_tab_type; + let js_model_type = getLongModelTypeFromShort(model_type); + let addedNodes = []; + + let is_thumb_mode = isThumbMode(getModelCardsEl(active_tab_type, js_model_type)); + + //metadata_buttoncard + metadata_button = card.querySelector(".metadata-button"); + //additional node + additional_node = card.querySelector(".actions .additional"); + //get ul node, which is the parent of all buttons + ul_node = card.querySelector(".actions .additional ul"); + + // check thumb mode + if (is_thumb_mode) { + additional_node.style.display = null; + + if (!ul_node) { + // nothing to do. + return; + } + + if (opts["ch_show_btn_on_thumb"]) { + ul_node.style.background = btn_thumb_background; + } else { + let ch_btn_txts = ["💡", "🌐", "🏷️", "✏️", "❌"]; + + // remove existed buttons + //reset + ul_node.style.background = null; + // find all .a child nodes + let atags = ul_node.querySelectorAll("a"); + + for (let atag of atags) { + //reset display + atag.style.display = null; + //remove extension's button + if (ch_btn_txts.indexOf(atag.textContent)>=0) { + //need to remove + ul_node.removeChild(atag); + } else { + //do not remove, just reset + atag.textContent = replace_preview_text; + atag.style.display = null; + atag.style.fontSize = null; + atag.style.position = null; + atag.style.backgroundImage = null; + } + } + + //also remove br tag in ul + let brtag = ul_node.querySelector("br"); + if (brtag) { + ul_node.removeChild(brtag); + } + + //just reset and remove nodes, do nothing else + return; + + } + + } else { + // full preview mode + + if (opts["ch_always_display"]) { + additional_node.style.display = "block"; + } else { + additional_node.style.display = null; + } + + if (!ul_node) { + ul_node = document.createElement("ul"); + } else { + // remove br tag + let brtag = ul_node.querySelector("br"); + if (brtag) { + ul_node.removeChild(brtag); + } + } + + } + + if (ul_node.dataset.ch_helper) { + return; + } + + ul_node.dataset.ch_helper = true; + + model_name = card.dataset.name; + + // replace preview text button + replace_preview_btn = card.querySelector(".actions .additional a"); + + if ((replace_preview_btn == null) && !("replace_preview_button" in opts["ch_hide_buttons"])) { + /* + * in sdwebui 1.5, the replace preview button has been + * moved to a hard to reach location, so we have to do + * quite a lot to get to its functionality. + */ + + // waste memory by keeping all of this in scope, per card. + let page = active_tab_type; + let type = js_model_type; + let name = card.dataset.name; + + // create the replace_preview_btn, as it no longer exists + replace_preview_btn = document.createElement("a"); + + // create an event handler to redirect a click to the real replace_preview_button + replace_preview_btn.setAttribute("onclick", `replace_preview(event, '${page}', '${type}', '${model_name}')`); + } + + // change replace preview text button into icon + if (!opts["ch_hide_buttons"].includes("replace_preview_button")) { + if (replace_preview_btn.textContent !== "🖼️") { + replace_preview_btn.textContent = "🖼️"; + addedNodes.push(replace_preview_btn); + } + + replace_preview_btn.classList.add("card-button", "removecard"); + + } else if (replace_preview_btn.parentElement) { + replace_preview_btn.parentElement.removeChild(replace_preview_btn); + } + + // search_term node + // search_term = subfolder path + model name + ext + search_term_node = card.querySelector(".actions .additional .search_term"); + if (!search_term_node) { + console.log("can not find search_term node for cards in " + active_tab_type + "_" + active_extra_tab_type + "_cards"); + return; + } + + // get search_term + search_term = search_term_node.textContent; + if (!search_term) { + console.log("search_term is empty for cards in " + active_tab_type + "_" + active_extra_tab_type + "_cards"); + return; + } + + // then we need to add buttons to each ul node: + if (!opts["ch_hide_buttons"].includes("open_url_button")) { + let open_url_node = document.createElement("a"); + open_url_node.href = "#"; + open_url_node.textContent = "🌐"; + open_url_node.classList.add("card-button", "openurl"); + open_url_node.title = "Open this model's civitai url"; + open_url_node.setAttribute("onclick", `open_model_url(event, '${model_type}', '${search_term}')`); + addedNodes.push(open_url_node); + } + + // add br if metadata_button exists + if (is_thumb_mode && metadata_button) { + addedNodes.push(document.createElement("br")); + } + + if (!opts["ch_hide_buttons"].includes("add_trigger_words_button")) { + let add_trigger_words_node = document.createElement("a"); + add_trigger_words_node.href = "#"; + add_trigger_words_node.textContent = "💡"; + add_trigger_words_node.classList.add("card-button", "addtriggerwords"); + add_trigger_words_node.title = "Add trigger words to prompt"; + add_trigger_words_node.setAttribute("onclick", `add_trigger_words(event, '${model_type}', '${search_term}')`); + addedNodes.push(add_trigger_words_node); + } + + if (!opts["ch_hide_buttons"].includes("add_preview_prompt_button")) { + let use_preview_prompt_node = document.createElement("a"); + use_preview_prompt_node.href = "#"; + use_preview_prompt_node.textContent = "🏷️"; + use_preview_prompt_node.classList.add("card-button", "usepreviewprompt"); + use_preview_prompt_node.title = "Use prompt from preview image"; + use_preview_prompt_node.setAttribute("onclick", `use_preview_prompt(event, '${model_type}', '${search_term}')`); + addedNodes.push(use_preview_prompt_node); + } + + if (!opts["ch_hide_buttons"].includes("rename_model_button")) { + let rename_card_node = document.createElement("a"); + rename_card_node.href = "#"; + rename_card_node.innerHTML = "✏️"; + rename_card_node.classList.add("card-button", "renamecard"); + rename_card_node.title = "Rename this model"; + rename_card_node.setAttribute("onclick", `rename_card(event, '${model_type}', '${search_term}', '${model_name}')`); + addedNodes.push(rename_card_node); + } + + if (!opts["ch_hide_buttons"].includes("remove_model_button")) { + let remove_card_node = document.createElement("a"); + remove_card_node.href = "#"; + remove_card_node.innerHTML = "❌"; + remove_card_node.classList.add("card-button", "removecard"); + remove_card_node.title = "Remove this model"; + remove_card_node.setAttribute("onclick", `remove_card(event, '${model_type}', '${search_term}')`); + addedNodes.push(remove_card_node); + } + + // add to buttons row + for (const node of addedNodes) { + ul_node.appendChild(node); + } + + // add buttons to card + if (!ul_node.parentElement && ul_node.children) { + additional_node.appendChild(ul_node); + } +} + +onUiLoaded(() => { + + //get gradio version + const gradio_ver = ch_gradio_version(); + console.log("Running Stable-Diffusion-Webui-Civitai-Helper on Gradio Version: " + gradio_ver); + + // get all extra network tabs + const tab_prefix_list = ["txt2img", "img2img"]; + const model_type_list = ["textual_inversion", "hypernetworks", "checkpoints", "lora", "lycoris"]; + + // update extra network tab pages' cards + // * replace "replace preview" text button into an icon + // * add 3 button to each card: + // - open model url 🌐 + // - add trigger words 💡 + // - use preview image's prompt 🏷️ + // notice: javascript can not get response from python side + // so, these buttons just sent request to python + // then, python side gonna open url and update prompt text box, without telling js side. + function update_card_for_civitai() { + if (!(opts && "ch_always_display" in opts)) { + // Lobe theme can cause a race condition. + return setTimeout(update_card_for_civitai, 500); + } + + let replace_preview_text = getTranslation("replace preview"); + if (!replace_preview_text) { + replace_preview_text = "replace preview"; + } + + let extra_network_node = null; + let model_type = ""; + let cards = null; + + //get current tab + let active_tab_type = getActiveTabType(); + if (!active_tab_type) {active_tab_type = "txt2img";} + + for (const tab_prefix of tab_prefix_list) { + if (tab_prefix != active_tab_type) {continue;} + + //get active extratab + const re = new RegExp(tab_prefix + "_(.+)_cards"); + const active_extra_tab = Array.from(get_uiCurrentTabContent().querySelectorAll('.extra-network-cards,.extra-network-thumbs')) + .find(el => el.closest('.tabitem').style.display === 'block') + ?.id.match(re)[1]; + + const active_extra_tab_type = getShortModelTypeFromFull(active_extra_tab); + + for (const js_model_type of model_type_list) { + //get model_type for python side + model_type = getShortModelTypeFromFull(js_model_type); + + if (!model_type) { + console.log("can not get model_type from: " + js_model_type); + continue; + } + + + //only handle current sub-tab + if (model_type != active_extra_tab_type) { + continue; + } + + //console.log("handle active extra tab"); + extra_network_id = tab_prefix + "_" + js_model_type + "_cards"; + + // console.log("searching extra_network_node: " + extra_network_id); + extra_network_node = getModelCardsEl(tab_prefix, js_model_type); + + // get all card nodes + cards = extra_network_node.querySelectorAll(".card"); + for (const card of cards) { + processSingleCard(active_tab_type, active_extra_tab_type, card); + } + + } + + } + + } + + /* + let extra_network_refresh_btn = null; + */ + let extra_networks_btn = null; + + //add refresh button to extra network's toolbar + for (const prefix of tab_prefix_list) { + // load extra networks button + extra_networks_btn = gradioApp().getElementById(prefix + "_extra_networks"); + + + // pre-1.6 + if (extra_networks_btn) { + function extraNetworksClick() { + waitForExtraTabs(prefix, model_type_list); + extra_networks_btn.removeEventListener("click", extraNetworksClick); + } + + // add listener to extra_networks_btn + extra_networks_btn.addEventListener("click", extraNetworksClick); + continue; + + } + + // 1.6 and higher + const extra_tab = getExtraTabs(prefix); + const headers = extra_tab.firstChild.children; + + for (const header of headers) { + const model_type = header.textContent.trim().replace(" ", "_").toLowerCase(); + + function extraNetworksClick() { + waitForExtraTabs(prefix, [model_type]); + header.removeEventListener("click", extraNetworksClick); + } + + header.addEventListener("click", extraNetworksClick); + } + + //get toolbar + extra_networks_btn = gradioApp().getElementById(prefix + "_extra_networks"); + + } + + //run it once + // update_card_for_civitai(); + + +}); + + + diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/__pycache__/civitai_helper.cpython-310.pyc b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/__pycache__/civitai_helper.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..244090007d8e064a13f3acbc109bcdc77b5e3465 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/__pycache__/civitai_helper.cpython-310.pyc differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__init__.py b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/__init__.cpython-310.pyc b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/__init__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..79cdcd1548020c7de7cf0858eb0beef631591462 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/__init__.cpython-310.pyc differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/civitai.cpython-310.pyc b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/civitai.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..6000279d97fd65df7336675956516307bcd4e7a5 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/civitai.cpython-310.pyc differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/downloader.cpython-310.pyc b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/downloader.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..4a9b1c8a15b08b5e42bf80c6e70e55010b3188d6 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/downloader.cpython-310.pyc differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/js_action_civitai.cpython-310.pyc b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/js_action_civitai.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..9608876353db4d533876a73530a235007d755950 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/js_action_civitai.cpython-310.pyc differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/model.cpython-310.pyc b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/model.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..e8372e93c0cac007a0eff482584b96d4a09e0dee Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/model.cpython-310.pyc differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/model_action_civitai.cpython-310.pyc b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/model_action_civitai.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..fdac75f018f300f17c6ea33d8ce4227af7d9da33 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/model_action_civitai.cpython-310.pyc differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/msg_handler.cpython-310.pyc b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/msg_handler.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..b7fd30322deccbe828195decdac4b71928d54892 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/msg_handler.cpython-310.pyc differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/templates.cpython-310.pyc b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/templates.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..fc6daff772ca317117512a54c494eef3f505b823 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/templates.cpython-310.pyc differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/util.cpython-310.pyc b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/util.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..aafd044a8201eeb2f8fad3f9a0a895d60bd7a658 Binary files /dev/null and b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/__pycache__/util.cpython-310.pyc differ diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/civitai.py b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/civitai.py new file mode 100644 index 0000000000000000000000000000000000000000..f515676aa05c0fe33675b42b4a2b24856c035dff --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/civitai.py @@ -0,0 +1,670 @@ +""" -*- coding: UTF-8 -*- +handle msg between js and python side +""" + +import os +import re +from . import util +from . import model +from . import downloader + +SUFFIX = ".civitai" + +URLS = { + "modelPage":"https://civitai.com/models/", + "modelId": "https://civitai.com/api/v1/models/", + "modelVersionId": "https://civitai.com/api/v1/model-versions/", + "hash": "https://civitai.com/api/v1/model-versions/by-hash/" +} + +MODEL_TYPES = { + "Checkpoint": "ckp", + "TextualInversion": "ti", + "Hypernetwork": "hyper", + "LORA": "lora", + "LoCon": "lycoris", +} + +NSFW_LEVELS = ["None", "Soft", "Mature", "X", "Allow All"] + +def civitai_get(civitai_url:str): + """ + Gets JSON from Civitai. + return: dict:json or None + """ + + success, response = downloader.request_get( + civitai_url + ) + + if not success: + return None + + # try to get content + content = None + try: + content = response.json() + except ValueError as e: + util.printD(util.indented_msg( + f""" + Parse response json failed + Error: {str(e)} + Response: {response.text} + """ + )) + return None + + return content + +def get_full_size_image_url(image_url, width): + """ + Get image with full size + Width is in number, not string + + return: url str + """ + return re.sub(r'/width=\d+/', '/width=' + str(width) + '/', image_url) + + +def append_parent_model_metadata(content): + """ + Some model metadata is stored in a "parent" context. + When we're fething a model by its hash, we're getting + the metadata for that model *file*, not the model entry + on Civitai, which may contain multiple versions. + + This method gets the parent metadata and appends it to + our model file metadata. + + return: model metadata with parent description, creator, + and permissions appended. + """ + util.printD("Fetching Parent Model Information") + parent_model = get_model_info_by_id(content["modelId"]) + + metadatas = [ + "description", "tags", "allowNoCredit", + "allowCommercialUse", "allowDerivatives", + "allowDifferentLicense" + ] + + content["creator"] = parent_model.get("creator", "{}") + + model_metadata = content["model"] + for metadata in metadatas: + model_metadata[metadata] = parent_model.get(metadata, "") + + return content + + +# use this sha256 to get model info from civitai +# return: model info dict +def get_model_info_by_hash(model_hash:str): + """ + use this sha256 to get model info from civitai's api + + return: + model info dict if a model is found + {} if civitai does not have the model + None if an error occurs. + """ + util.printD("Request model info from civitai") + + if not model_hash: + util.printD("hash is empty") + return None + + content = civitai_get(f'{URLS["hash"]}{model_hash}') + + if content: + content = append_parent_model_metadata(content) + + return content + + +def get_model_info_by_id(model_id:str) -> dict: + """ + Fetches model info by its model id. + returns: dict:model_info + """ + + util.printD(f"Request model info from civitai: {model_id}") + + if not model_id: + util.printD("model_id is empty") + return False + + content = civitai_get(f'{URLS["modelId"]}{model_id}') + + return content + + +def get_version_info_by_version_id(version_id:str) -> dict: + """ + Gets model version info from Civitai by version id + return: dict:model_info + """ + util.printD("Request version info from civitai") + + if not version_id: + util.printD("version_id is empty") + return None + + content = civitai_get(f'{URLS["modelVersionId"]}{version_id}') + + if content: + content = append_parent_model_metadata(content) + + return content + + +def get_version_info_by_model_id(model_id:str) -> dict: + """ + Fetches version info by model id. + returns: dict:version_info + """ + + model_info = get_model_info_by_id(model_id) + if not model_info: + util.printD(f"Failed to get model info by id: {model_id}") + return None + + # check content to get version id + versions = model_info.get("modelVersions", []) + if len(versions) == 0: + util.printD("Found no model versions") + return None + + def_version = versions[0] + if not def_version: + util.printD("default version is None") + return None + + version_id = def_version.get("id", "") + + if not version_id: + util.printD("Could not get valid version id") + return None + + # get version info + version_info = get_version_info_by_version_id(f"{version_id}") + if not version_info: + util.printD(f"Failed to get version info by version_id: {version_id}") + return None + + return version_info + + +def load_model_info_by_search_term(model_type, search_term): + """ + get model info file's content by model type and search_term + parameter: model_type, search_term + return: model_info + """ + util.printD(f"Load model info of {search_term} in {model_type}") + if model.folders.get(model_type, None) is None: + util.printD(f"unknown model type: {model_type}") + return None + + # search_term = f"{subfolderpath}{model name}{ext}" + # And it always start with a / even when there is no sub folder + base, _ = os.path.splitext(search_term) + model_info_base = base + if base[:1] == "/": + model_info_base = base[1:] + + if model_type == "lora" and model.folders['lycoris']: + model_folders = [model.folders[model_type], model.folders['lycoris']] + else: + model_folders = [model.folders[model_type]] + + for model_folder in model_folders: + model_info_filename = f"{model_info_base}{SUFFIX}{model.CIVITAI_EXT}" + model_info_filepath = os.path.join(model_folder, model_info_filename) + + found = os.path.isfile(model_info_filepath) + + if found: + break + + if not found: + util.printD(f"Can not find model info file: {model_info_filepath}") + return None + + return model.load_model_info(model_info_filepath) + + +def get_model_names_by_type_and_filter(model_type:str, metadata_filter:dict) -> list: + """ + get model file names by model type + parameter: model_type - string + parameter: filter - dict, which kind of model you need + return: model name list + """ + + if model_type == "lora" and model.folders['lycoris']: + model_folders = [model.folders[model_type], model.folders['lycoris']] + else: + model_folders = [model.folders[model_type]] + + # set metadata_filter + # only get models don't have a civitai info file + no_info_only = False + empty_info_only = False + + if metadata_filter: + no_info_only = metadata_filter.get("no_info_only", False) + empty_info_only = metadata_filter.get("empty_info_only", False) + + # get information from filter + # only get those model names don't have a civitai model info file + model_names = [] + for model_folder in model_folders: + for root, _, files in os.walk(model_folder, followlinks=True): + for filename in files: + if is_valid_file(root, filename, no_info_only, empty_info_only): + model_names.append(filename) + + return model_names + + +def is_valid_file(root, filename, no_info_only, empty_info_only): + """ + Filters through model files to determine if they are + valid targets for downloading new metadata. + + return: bool + """ + item = os.path.join(root, filename) + # check extension + base, ext = os.path.splitext(item) + if ext not in model.EXTS: + return False + + # find a model + info_file = f"{base}{SUFFIX}{model.CIVITAI_EXT}" + + # check filter + if os.path.isfile(info_file): + if no_info_only: + return False + + if empty_info_only: + # load model info + model_info = model.load_model_info(info_file) + # check content + if model_info and "id" in model_info.keys(): + # find a non-empty model info file + return False + + return True + + +def get_model_names_by_input(model_type, empty_info_only): + """ return: list of model filenames with empty civitai info files """ + return get_model_names_by_type_and_filter(model_type, {"empty_info_only":empty_info_only}) + + +# get id from url +def get_model_id_from_url(url:str) -> str: + """ return: model_id from civitai url """ + util.printD("Run get_model_id_from_url") + model_id = "" + + if not url: + util.printD("url or model id can not be empty") + return "" + + if url.isnumeric(): + # is already an model_id + model_id = f"{url}" + return model_id + + split_url = re.sub("\\?.+$", "", url).split("/") + if len(split_url) < 2: + util.printD("url is not valid") + return "" + + if split_url[-2].isnumeric(): + model_id = split_url[-2] + elif split_url[-1].isnumeric(): + model_id = split_url[-1] + else: + util.printD("There is no model id in this url") + return "" + + return model_id + + +def preview_exists(model_path): + """ Search for existing preview image. return True if it exists, else false """ + + previews = model.get_potential_model_preview_files(model_path) + + for prev in previews: + if os.path.isfile(prev): + return True + + return False + + +def should_skip(user_rating, image_rating): + """ return: True if preview_nsfw level higher than user threshold """ + order = NSFW_LEVELS + return order.index(image_rating) >= order.index(user_rating) + + +def verify_preview(path, img_dict, max_size_preview, nsfw_preview_threshold): + """ + Downloads a preview image if it meets the user's requirements. + """ + + img_url = img_dict.get("url", None) + if img_url is None: + yield (False, None) + + image_rating = img_dict.get("nsfw", "None") + if image_rating != "None": + util.printD(f"This image is NSFW: {image_rating}") + if should_skip(nsfw_preview_threshold, image_rating): + util.printD("Skip NSFW image") + yield (False, None) + + if max_size_preview: + # use max width + if "width" in img_dict.keys(): + if img_dict["width"]: + img_url = get_full_size_image_url(img_url, img_dict["width"]) + + success = False + preview_path = "" + for result in downloader.dl_file(img_url, file_path=path): + if not isinstance(result, str): + success, preview_path = result + break + + yield result + + + if not success: + yield (False, None) + + # we only need 1 preview image + yield (True, preview_path) + + +# get preview image by model path +# image will be saved to file, so no return +def get_preview_image_by_model_path(model_path:str, max_size_preview, nsfw_preview_threshold): + """ + Downloads a preview image for a model if one doesn't already exist. + Skips images that are more NSFW than the user's NSFW threshold + """ + util.printD("Downloading model image.") + + if not model_path: + util.printD("model_path is empty") + return + + if not os.path.isfile(model_path): + util.printD(f"model_path is not a file: {model_path}") + return + + base, _ = os.path.splitext(model_path) + preview_path = f"{base}.preview.png" # TODO png not strictly required + info_file = f"{base}{SUFFIX}{model.CIVITAI_EXT}" + + # need to download preview image + util.printD(f"Checking preview image for model: {model_path}") + + if preview_exists(model_path): + util.printD("Existing model image found. Skipping.") + return + + # load model_info file + if not os.path.isfile(info_file): + return + + try: + images = model.load_model_info(info_file)["images"] + + except (KeyError, TypeError): + return + + success = False + for img_dict in images: + for result in verify_preview( + preview_path, img_dict, max_size_preview, nsfw_preview_threshold + ): + if not isinstance(result, str): + success, _ = result + # Only download one image + if success: + return + + break + + yield result + + util.printD(f"Could not find any valid preview images for model: {model_path}") + yield + + +# search local model by version id in 1 folder, no subfolder +# return - model_info +def search_local_model_info_by_version_id(folder:str, version_id:int) -> dict: + """ Searches a folder for model_info files, + returns the model_info from a file if its id matches the model id. + """ + util.printD("Searching local model by version id") + util.printD(f"folder: {folder}") + util.printD(f"version_id: {version_id}") + + if not folder: + util.printD("folder is none") + return None + + if not os.path.isdir(folder): + util.printD("folder is not a dir") + return None + + if not version_id: + util.printD("version_id is none") + return None + + # search civitai model info file + for filename in os.listdir(folder): + # check ext + base, ext = os.path.splitext(filename) + if ext == model.CIVITAI_EXT: + # find info file + if not (len(base) > 8 and base[-8:] == SUFFIX): + # not a civitai info file + continue + + # find a civitai info file + path = os.path.join(folder, filename) + model_info = model.load_model_info(path) + if not model_info: + continue + + model_id = model_info.get("id", None) + if not model_id: + continue + + # util.printD(f"Compare version id, src: {model_id}, target:{version_id}") + if f"{model_id}" == f"{version_id}": + # find the one + return model_info + + return None + + +def get_model_id_from_model_path(model_path:str): + """ return model_id using model_path """ + # get model info file name + base, _ = os.path.splitext(model_path) + info_file = f"{base}{SUFFIX}{model.CIVITAI_EXT}" + + if not os.path.isfile(info_file): + return None + + # get model info + model_info_file = model.load_model_info(info_file) + local_version_id = model_info_file.get("id", None) + model_id = model_info_file.get("modelId", None) + + if None in [model_id, local_version_id]: + return None + + return (model_id, local_version_id) + + +def check_model_new_version_by_path(model_path:str, delay:float=0.2) -> tuple: + """ + check new version for a model by model path + return ( + model_path, model_id, model_name, new_verion_id, + new_version_name, description, download_url, img_url + ) + """ + + if not (model_path and os.path.isfile(model_path)): + util.printD(f"model_path is not a file: {model_path}") + return None + + result = get_model_id_from_model_path(model_path) + if not result: + return None + + model_id, local_version_id = result + + # get model info by id from civitai + model_info = get_model_info_by_id(model_id) + + util.delay(delay) + + if not model_info: + return None + + model_versions = model_info.get("modelVersions", []) + + if len(model_versions) == 0: + return None + + current_version = model_versions[0] + if not current_version: + return None + + current_version_id = current_version.get("id", False) + + util.printD(f"Compare version id, local: {local_version_id}, remote: {current_version_id}") + + if not (current_version_id and current_version_id != local_version_id): + return None + + model_name = model_info.get("name", "") + new_version_name = current_version.get("name", "") + description = current_version.get("description", "") + download_url = current_version.get("downloadUrl", "") + + # get 1 preview image + try: + img_url = current_version["images"][0]["url"] + except (IndexError, KeyError): + img_url = "" + + return ( + model_path, model_id, model_name, current_version_id, + new_version_name, description, download_url, img_url + ) + + +def check_single_model_new_version(root, filename, model_type, delay): + """ + return: True if a valid model has a new version. + """ + # check ext + item = os.path.join(root, filename) + _, ext = os.path.splitext(item) + + if ext not in model.EXTS: + return False + + # find a model + request = check_model_new_version_by_path(item, delay) + + if not request: + return False + + request = request + (model_type,) + + # model_path, model_id, model_name, version_id, new_version_name, description, downloadUrl, img_url = request + version_id = request[3] + + # check exist + if not version_id: + return False + + # search this new version id to check if this model is already downloaded + target_model_info = search_local_model_info_by_version_id(root, version_id) + if target_model_info: + util.printD("New version is already exists") + return False + + return request + + +def check_models_new_version_by_model_types(model_types:list, delay:float=0.2) -> list: + """ + check all models of model_types for new version + parameter: delay - float, how many seconds to delay between each request to civitai + return: new_versions + a list for all new versions, each one is + (model_path, model_id, model_name, new_verion_id, + new_version_name, description, download_url, img_url) + """ + util.printD("Checking models' new version") + + if not model_types: + return [] + + # check model types, which could be a string as 1 type + mts = [] + if isinstance(model_types, str): + mts.append(model_types) + elif isinstance(model_types, list): + mts = model_types + else: + util.printD("Unknown model types:") + util.printD(model_types) + return [] + + # new version list + new_versions = [] + new_version_ids = [] + + # walk all models + for model_type, model_folder in model.folders.items(): + if model_type not in mts: + continue + + util.printD(f"Scanning path: {model_folder}") + for root, _, files in os.walk(model_folder, followlinks=True): + for filename in files: + version = check_single_model_new_version(root, filename, model_type, delay) + + if not version: + continue + + # model_path, model_id, model_name, version_id, new_version_name, description, downloadUrl, img_url = version + version_id = version[3] + + if version_id in new_version_ids: + continue + + # add to list + new_versions.append(version) + new_version_ids.append(version_id) + + return new_versions diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/downloader.py b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/downloader.py new file mode 100644 index 0000000000000000000000000000000000000000..1fbc78723da8774afaec9ebbec347ba6fd45d15b --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/downloader.py @@ -0,0 +1,313 @@ +""" -*- coding: UTF-8 -*- +Used for downloading files +""" +from __future__ import annotations +import os +import platform +import time +from tqdm import tqdm +import requests +import urllib3 +from . import util + + +DL_EXT = ".downloading" +MAX_RETRIES = 3 + +# disable ssl warning info +urllib3.disable_warnings() + + +def request_get(url:str, headers=None, retries=0) -> tuple[bool, requests.Response]: + """ + Performs a GET request + return: request + """ + + headers = util.append_default_headers(headers or {}) + + try: + response = requests.get( + url, + stream=True, + verify=False, + headers=headers, + proxies=util.PROXIES, + timeout=util.REQUEST_TIMEOUT + ) + + except TimeoutError: + output = f"GET Request timed out for {url}" + print(output) + return (False, output) + + if not response.ok: + code = response.status_code + reason = response.reason + util.printD(util.indented_msg( + f""" + GET Request failed with error code: + {code}: {reason} + """ + )) + + if response.status_code == 401: + return ( + False, + "This download requires Authentication. Please add an API Key to Civitai Helper's settings to continue this download. See [Wiki](https://github.com/zixaphir/Stable-Diffusion-Webui-Civitai-Helper/wiki/Civitai-API-Key) for details on how to create an API Key." + ) + + if response.status_code != 404 and retries < MAX_RETRIES: + util.printD("Retrying") + return request_get(url, headers, retries + 1) + + return (False, response) + + return (True, response) + + +def visualize_progress(percent:int, downloaded, total, speed, show_bar=True) -> str: + """ Used to display progress in webui """ + + percent_as_int = percent + total = f"{total}" + downloaded = f"{downloaded:>{len(total)}}" + percent = f"{percent:>3}" + + snippet = f"`{percent}%: {downloaded} / {total} @ {speed}`" + + if not show_bar: + # Unfortunately showing a progress bar in webui + # is very weird on mobile with limited horizontal + # space + return snippet.replace(" ", "\u00a0") + + progress = "\u2588" * percent_as_int + + return f"`[{progress:<100}] {snippet}`".replace(" ", "\u00a0") + + +def download_progress(url:str, file_path:str, total_size:int, headers={}) -> bool | float: + """ + Performs a file download. + returns: True or an error message. + """ + # use a temp file for downloading + + dl_path = f"{file_path}{DL_EXT}" + + util.printD(f"Downloading to temp file: {dl_path}") + + # check if downloading file exists + downloaded_size = 0 + if os.path.exists(dl_path): + downloaded_size = os.path.getsize(dl_path) + util.printD(f"Resuming partially downloaded file from progress: {downloaded_size}") + + # create header range + headers["Range"] = f"bytes={downloaded_size:d}-" + headers = util.append_default_headers(headers) + + # download with header + success, response = request_get( + url, + headers=headers, + ) + + if not success: + yield (False, response) + + last_tick = 0 + start = time.time() + + downloaded_this_session = 0 + + # write to file + with open(dl_path, 'wb') as target, tqdm( + total=total_size, + unit='iB', + unit_scale=True, + unit_divisor=1024 + ) as progress_bar: + for chunk in response.iter_content(chunk_size=256*1024): + if chunk: + downloaded_this_session += len(chunk) + downloaded_size += len(chunk) + written = target.write(chunk) + + # write to disk + target.flush() + + progress_bar.update(written) + + percent = int(100 * (downloaded_size / total_size)) + timer = time.time() + + # Gradio output is a *slooowwwwwwww* asynchronous FIFO queue + if timer - last_tick > 0.2 or percent == 100: + + last_tick = timer + elapsed = timer - start + speed = downloaded_this_session // elapsed if elapsed >= 1 \ + else downloaded_this_session + + # Mac reports filesizes in multiples of 1000 + # We should respect platform differences + unit = 1000 if platform.system() == "Darwin" else 1024 + + i = 0 + while speed > unit: + i += 1 + speed = speed / unit + if i >= 3: + break + + speed = f'{round(speed, 2)}{["", "K", "M", "G"][i]}Bps' + + text_progress = visualize_progress( + percent, + downloaded_size, + total_size, + speed, + False + ) + + yield text_progress + + # check file size + downloaded_size = os.path.getsize(dl_path) + if downloaded_size != total_size: + warning = util.indented_msg( + f""" + File is not the correct size: {file_path}. + Expected {total_size:d}, got {downloaded_size:d}. + The file may be corrupt. If you encounter issues, + you can try again later or download the file manually: {url} + """ + ) + util.warning(warning) + util.printD(warning) + + # rename file + os.rename(dl_path, file_path) + output = f"File Downloaded to: {file_path}" + util.printD(output) + + yield (True, file_path) + + +def get_file_path_from_service_headers(response:requests.Response, folder:str) -> str: + """ + Parses a response header to get a filename + then builds a file_path. + + return: file_path:str + """ + + content_disposition = response.headers.get("Content-Disposition", None) + + if content_disposition is None: + util.printD("Can not get file name from download url's header") + return None + + # Extract the filename from the header + # content of a CD: "attachment;filename=FileName.txt" + # in case "" is in CD filename's start and end, need to strip them out + filename = content_disposition.split("=")[1].strip('"') + filename = filename.encode('iso8859-1').decode('utf-8') + if not filename: + util.printD(f"Fail to get file name from Content-Disposition: {content_disposition}") + return None + + # with folder and filename, now we have the full file path + return os.path.join(folder, filename) + + +# output is downloaded file path +def dl_file( + url:str, + folder=None, + filename=None, + file_path=None, + headers={}, + duplicate=None +) -> tuple[bool, str]: + """ + Perform a download. + + returns: tuple(success:bool, filepath or failure message:str) + """ + + success, response = request_get(url, headers=headers) + + if not success: + yield (False, response) + + util.printD(f"Start downloading from: {url}") + + # get file_path + if not file_path: + if not (folder or os.path.isdir(folder)): + yield ( + False, + "No directory to save model to." + ) + + if filename: + file_path = os.path.join(folder, filename) + else: + file_path = get_file_path_from_service_headers(response, folder) + + if not file_path: + yield ( + False, + f"Could not get a file_path to place saved file:" + ) + + util.printD(f"Target file path: {file_path}") + base, ext = os.path.splitext(file_path) + + # duplicate handling + if os.path.isfile(file_path): + if duplicate == "Rename New": + # check if file is already exist + count = 2 + new_base = base + while os.path.isfile(file_path): + util.printD("Target file already exist.") + # rename duplicate + new_base = f"{base}_{count}" + file_path = f"{new_base}{ext}" + count += 1 + + elif duplicate != "Overwrite": + yield ( + False, + f"File {file_path} already exists! Download will not proceed." + ) + + # get file size + total_size = int(response.headers['Content-Length']) + util.printD(f"File size: {total_size}") + + for result in download_progress(url, file_path, total_size, headers): + if not isinstance(result, str): + success, output = result + break + + yield result + + yield (success, output) + + +def error(download_url:str, msg:str) -> str: + """ Display a download error """ + output = util.indented_msg( + f""" + Download failed. + {msg} + Download url: {download_url} + """ + ) + util.printD(output) + return output diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/js_action_civitai.py b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/js_action_civitai.py new file mode 100644 index 0000000000000000000000000000000000000000..c8b54f1b0ebf4e7e8618be7aac7ed0fce9bd885e --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/js_action_civitai.py @@ -0,0 +1,379 @@ +""" -*- coding: UTF-8 -*- +handle msg between js and python side +""" +import os +from pathlib import Path +import webbrowser +from . import util +from . import model +from . import civitai +from . import msg_handler +from . import downloader + + +def open_model_url(msg): + """ + get civitai's model url and open it in browser + parameter: model_type, search_term + output: python msg + - will be sent to hidden textbox then picked by js side + """ + util.printD("Start open_model_url") + + output = "" + result = msg_handler.parse_js_msg(msg) + if not result: + util.printD("Parsing js ms failed") + return None + + model_type = result["model_type"] + search_term = result["search_term"] + + model_info = civitai.load_model_info_by_search_term(model_type, search_term) + if not model_info: + util.printD(f"Failed to get model info for {model_type} {search_term}") + return "" + + if "modelId" not in model_info.keys(): + util.printD(f"Failed to get model id from info file for {model_type} {search_term}") + return "" + + model_id = model_info["modelId"] + if not model_id: + util.printD(f"model id from info file of {model_type} {search_term} is None") + return "" + + url = f'{civitai.URLS["modelPage"]}{model_id}' + + # msg content for js + content = { + "url": "" + } + + if not util.get_opts("ch_open_url_with_js"): + util.printD(f"Open Url: {url}") + # open url + webbrowser.open_new_tab(url) + else: + util.printD("Send Url to js") + content["url"] = url + output = msg_handler.build_py_msg("open_url", content) + + util.printD("End open_model_url") + return output + + +def add_trigger_words(msg): + """ + add trigger words to prompt + parameter: model_type, search_term, prompt + return: [new_prompt, new_prompt] + - new prompt with trigger words, return twice for txt2img and img2img + """ + util.printD("Start add_trigger_words") + + result = msg_handler.parse_js_msg(msg) + if not result: + util.printD("Parsing js ms failed") + return None + + model_type = result["model_type"] + search_term = result["search_term"] + prompt = result["prompt"] + + model_info = civitai.load_model_info_by_search_term(model_type, search_term) + if not model_info: + util.printD(f"Failed to get model info for {model_type} {search_term}") + return [prompt, prompt] + + if "trainedWords" not in model_info.keys(): + util.printD(f"Failed to get trainedWords from info file for {model_type} {search_term}") + return [prompt, prompt] + + trained_words = model_info.get("trainedWords", []) + if len(trained_words) == 0: + util.printD(f"trainedWords from info file for {model_type} {search_term} is empty") + return [prompt, prompt] + + # guess if trained words are a list of words or list of prompts + prompt_list = ',' in trained_words[0] + + # if a list of prompts, join with a newline, else a comma and a space + separator = "\n" if prompt_list else ", " + trigger_words = separator.join(trained_words) + + new_prompt = f"{prompt} {trigger_words}" + + util.printD(f"trigger_words: {trigger_words}") + util.printD(f"prompt: {prompt}") + util.printD(f"new_prompt: {new_prompt}") + + util.printD("End add_trigger_words") + + # add to prompt + return [new_prompt, new_prompt] + + +def use_preview_image_prompt(msg): + """ + use preview image's prompt as prompt + parameter: model_type, model_name, prompt, neg_prompt + return: [new_prompt, new_neg_prompt, new_prompt, new_neg_prompt,] + - return twice for txt2img and img2img + """ + util.printD("Start use_preview_image_prompt") + + result = msg_handler.parse_js_msg(msg) + if not result: + util.printD("Parsing js ms failed") + return None + + model_type = result["model_type"] + search_term = result["search_term"] + prompt = result["prompt"] + neg_prompt = result["neg_prompt"] + + + model_info = civitai.load_model_info_by_search_term(model_type, search_term) + if not model_info: + util.printD(f"Failed to get model info for {model_type} {search_term}") + return [prompt, neg_prompt, prompt, neg_prompt] + + images = model_info.get("images", []) + if len(images) == 0: + util.printD(f"No images from info file for {model_type} {search_term}") + return [prompt, neg_prompt, prompt, neg_prompt] + + # get prompt from preview images' meta data + preview_prompt = "" + preview_neg_prompt = "" + for img in images: + meta = img.get("meta", {}) + preview_prompt = meta.get("prompt", "") + preview_neg_prompt = meta.get("negativePrompt", "") + + # we only need 1 prompt + if preview_prompt: + break + + if not preview_prompt: + util.printD(f"There is no prompt of {model_type} {search_term} in its preview image") + return [prompt, neg_prompt, prompt, neg_prompt] + + util.printD("End use_preview_image_prompt") + + return [preview_prompt, preview_neg_prompt, preview_prompt, preview_neg_prompt] + + +def dl_model_new_version(msg, max_size_preview, nsfw_preview_threshold): + """ + download model's new verson by model path, version id and download url + output is a md log + + This method is triggered by a click event on the client/js + side that sends a signal to download a single new model + version. The actual check for new models is in + `model_action_civitai.check_models_new_version_to_md`. + return: output:str + """ + util.printD("Start dl_model_new_version") + + output = "" + + result = msg_handler.parse_js_msg(msg) + if not result: + output = "Parsing js msg failed" + util.printD(output) + yield output + return + + model_path = result["model_path"] + version_id = result["version_id"] + download_url = result["download_url"] + model_type = result["model_type"] + + # check data + if not (model_path and version_id and download_url): + output = util.indented_msg(f""" + Missing parameter: + {model_path=} + {version_id=} + {download_url=} + """) + util.printD(output) + yield output + return + + util.printD(f"model_path: {model_path}") + util.printD(f"version_id: {version_id}") + util.printD(f"download_url: {download_url}") + + if not os.path.isfile(model_path): + output = f"model_path is not a file: {model_path}" + util.printD(output) + yield output + return + + # get model folder from model path + model_folder = os.path.dirname(model_path) + + success = False + # download file + webui visible progress bar + for result in downloader.dl_file(download_url, folder=model_folder): + if not isinstance(result, str): + success, output = result + break + + yield result + + if not success: + util.printD(output) + yield "Model download failed. See console for more details." + return + + # get version info + version_info = civitai.get_version_info_by_version_id(version_id) + + # now write version info to files + model.process_model_info(output, version_info, model_type) + + # then, get preview image + for result in civitai.get_preview_image_by_model_path( + output, + max_size_preview, + nsfw_preview_threshold + ): + yield result + + output = f"Done. Model downloaded to: {output}" + util.printD(output) + yield output + + +def get_model_path_from_js_msg(result): + """ + Gets a model path based on the webui js_msg. + + return: model_path + """ + if not result: + output = "Parsing js ms failed" + util.error(output) + util.printD(output) + return None + + model_type = result["model_type"] + search_term = result["search_term"] + + model_path = model.get_model_path_by_search_term(model_type, search_term) + if not model_path: + output = f"Fail to get model for {model_type} {search_term}" + util.error(output) + util.printD(output) + return None + + if not os.path.isfile(model_path): + output = f"Model {model_type} {search_term} does not exist, no need to remove" + util.error(output) + util.printD(output) + return None + + return model_path + + +def make_new_filename(candidate_file, model_name, new_name): + """ + Substitutes and old model name for a new model name. + return: new_path:str or None + """ + path, filename = os.path.split(candidate_file) + + if filename.index(model_name) != 0: + output = util.indented_msg(f""" + Could not find model_name in candidate file + {model_name=} + {candidate_file=} + {new_name=} + """) + util.error(output) + util.printD(output) + return None + + # handles [model_name].civitai.info and [model_name].preview.[ext] + new_filename = filename.replace(model_name, new_name, 1) + + new_path = os.path.join(path, new_filename) + + return new_path + + +def rename_model_by_path(msg): + """ + Rename a model file and all related ch_helper/ + preview image files. + """ + util.printD("Start rename_model_by_path") + + output = "" + result = msg_handler.parse_js_msg(msg) + + model_path = get_model_path_from_js_msg(result) + + if model_path is None: + output = "Could not rename model." + return output + + # all files need to be renamed + model_files = model.get_model_files_from_model_path(model_path) + model_name = Path(model_path).stem + new_name = util.bash_filename(result["new_name"]) + + renamed = [] + for candidate_file in model_files: + new_path = make_new_filename(candidate_file, model_name, new_name) + if new_path is None: + continue + + renamed.append(f"* {candidate_file} to {new_path}") + util.printD(f"Renaming file {candidate_file} to {new_path}") + os.rename(candidate_file, new_path) + + renamed = "\n".join(renamed) + status = f"The following files were renamed: \n{renamed}" + util.info(status) + + util.printD("End rename_model_by_path") + return output + + +def remove_model_by_path(msg): + """ + Remove a model file and all related ch_helper/ + preview image files. + """ + output = "" + util.printD("Start remove_model_by_path") + + result = msg_handler.parse_js_msg(msg) + + model_path = get_model_path_from_js_msg(result) + + if model_path is None: + output = "Could not remove model." + return output + + # all files need to be renamed + model_files = model.get_model_files_from_model_path(model_path) + + removed = [] + for candidate_file in model_files: + util.printD(f"* Removing file {candidate_file}") + removed.append(candidate_file) + os.remove(candidate_file) + + removed = "\n".join(removed) + status = f"The following files were removed: \n{removed}" + util.info(status) + + util.printD("End remove_model_by_path") + return output diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/model.py b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/model.py new file mode 100644 index 0000000000000000000000000000000000000000..f738bf97283638e6bf26f03c1f321f4f435b3b88 --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/model.py @@ -0,0 +1,663 @@ +""" -*- coding: UTF-8 -*- +Handle model operations +""" +import os +import json +import re +from PIL import Image +import piexif +import piexif.helper +from modules import shared +from modules import paths_internal +from . import civitai +from . import downloader +from . import util + + +# this is the default root path +ROOT_PATH = paths_internal.data_path + +EXTS = (".bin", ".pt", ".safetensors", ".ckpt") +CIVITAI_EXT = ".info" +SDWEBUI_EXT = ".json" + +""" +If command line arguement is used to change model folder, +then model folder is in absolute path, not based on this root path anymore. +so to make extension work with those absolute model folder paths, model +folder also need to be in absolute path +""" +folders = { + "ti": os.path.join(ROOT_PATH, "embeddings"), + "hyper": os.path.join(ROOT_PATH, "models", "hypernetworks"), + "ckp": os.path.join(ROOT_PATH, "models", "Stable-diffusion"), + "lora": os.path.join(ROOT_PATH, "models", "Lora"), + "lycoris": os.path.join(ROOT_PATH, "models", "LyCORIS"), +} + + +class VersionMismatchException(Exception): + """ Used for version comarison failures """ + + def __init__(self, value): + self.value = value + + def __str__(self): + return repr(self.value) + + +def get_model_info_paths(model_path): + """ + Retrieve model info paths + return: (info_file:str, sd15_file:str) + """ + base, _ = os.path.splitext(model_path) + info_file = f"{base}{civitai.SUFFIX}{CIVITAI_EXT}" + sd15_file = f"{base}{SDWEBUI_EXT}" + return (info_file, sd15_file) + + +# get custom model path +def get_custom_model_folder(): + """ + Update extra network directories with user-specified values. + """ + util.printD("Get Custom Model Folder") + + if shared.cmd_opts.embeddings_dir and os.path.isdir(shared.cmd_opts.embeddings_dir): + folders["ti"] = shared.cmd_opts.embeddings_dir + + if shared.cmd_opts.hypernetwork_dir and os.path.isdir(shared.cmd_opts.hypernetwork_dir): + folders["hyper"] = shared.cmd_opts.hypernetwork_dir + + if shared.cmd_opts.ckpt_dir and os.path.isdir(shared.cmd_opts.ckpt_dir): + folders["ckp"] = shared.cmd_opts.ckpt_dir + + if shared.cmd_opts.lora_dir and os.path.isdir(shared.cmd_opts.lora_dir): + folders["lora"] = shared.cmd_opts.lora_dir + + try: + # pre-1.5.0 + if os.path.isdir(shared.cmd_opts.lyco_dir): + folders["lycoris"] = shared.cmd_opts.lyco_dir + + except AttributeError: + try: + # sd-webui v1.5.1 added a backcompat option for lyco. + if os.path.isdir(shared.cmd_opts.lyco_dir_backcompat): + folders["lycoris"] = shared.cmd_opts.lyco_dir_backcompat + + except AttributeError: + # v1.5.0 has no options for the Lyco dir: + # it is hardcoded as 'os.path.join(paths.models_path, "LyCORIS")' + return + + +def metadata_needed(info_file, sd15_file, refetch_old): + """ return True if metadata is needed + """ + + need_civitai = metadata_needed_for_type(info_file, "civitai", refetch_old) + need_sdwebui = metadata_needed_for_type(sd15_file, "sdwebui", refetch_old) + + return need_civitai or need_sdwebui + + +def metadata_needed_for_type(path, meta_type, refetch_old): + """ return True if metadata is needed for path + """ + + if meta_type == "sdwebui" and not util.get_opts("ch_dl_webui_metadata"): + return False + + if not os.path.isfile(path): + return True + + if refetch_old: + metadata = None + with open(path) as file: + metadata = json.load(file) + + metadata_version = util.metadata_version(metadata) + + if not metadata_version: + return True + + if meta_type == "civitai": + compat_version = util.COMPAT_VERSION_CIVITAI + else: + compat_version = util.COMPAT_VERSION_SDWEBUI + + util.printD(f"{path}: {metadata_version}, {compat_version}") + + return util.newer_version(compat_version, metadata_version) + + return False + + +def verify_overwrite_eligibility(path, new_data): + """ + Verifies a file is valid to be overwritten + Throws an error if the model ID does not match the new version's model ID + return: True if valid, False if not. + """ + if not os.path.isfile(path): + return True + + with open(path, "r") as file: + old_data = json.load(file) + + if "civitai" in path: + new_id = new_data.get("id", "") + old_id = old_data.get("id", "") + if new_id != old_id: + if old_id != "": + raise VersionMismatchException( + f"New metadata id ({new_id}) does not match old metadata id ({old_id})" + ) + + new_description = new_data.get("description", "") + old_description = old_data.get("description", "") + if new_description == "" and old_description != "": + util.printD( + f"New description is blank while old description contains data. Skipping {path}" + ) + return False + + return True + + +def write_info(data, path, info_type): + """ Writes model info to a file """ + util.printD(f"Write model {info_type} info to file: {path}") + with open(os.path.realpath(path), 'w') as info_file: + info_file.write(json.dumps(data, indent=4)) + + +def process_model_info(model_path, model_info, model_type="ckp", refetch_old=False): + """ + Write model info to file + + SD1.5 Webui added saving model information to JSON files. + Much of this extension's metadata management is replicated + by this new functionality, including automatically adding + activator keywords to the prompt. It also provides a much + cleaner UI than civitai (not a high bar to clear) to + simply read a model's description. + + So why not populate it with useful information? + + Returns True if successful, otherwise an error message. + """ + + if model_info is None: + util.printD("Failed to get model info.") + return + + info_file, sd15_file = get_model_info_paths(model_path) + + parent = model_info["model"] + + description = parent.get("description", "") + if description: + description = util.trim_html(description) + parent["description"] = description + + version_description = model_info.get("description", "") + if version_description: + version_description = util.trim_html(version_description) + model_info["description"] = version_description + + tags = parent.get("tags", []) + parent["tags"] = tags + + # Create extension versioning information so that users + # can replace stale info files without newer entries. + model_info["extensions"] = util.create_extension_block(model_info.get("extensions", {})) + + # civitai model info file + if metadata_needed_for_type(info_file, "civitai", refetch_old): + if refetch_old: + try: + if verify_overwrite_eligibility(info_file, model_info): + write_info(model_info, info_file, "civitai") + except VersionMismatchException as e: + util.printD(f"{e}, aborting") + return + else: + write_info(model_info, info_file, "civitai") + + if not util.get_opts("ch_dl_webui_metadata"): + return + + # Do not overwrite user-created files! + # TODO: maybe populate empty fields in existing files? + if not metadata_needed_for_type(sd15_file, "sdwebui", refetch_old): + util.printD(f"Metadata not needed for: {sd15_file}.") + return + + process_sd15_info(sd15_file, model_info, parent, model_type, refetch_old) + + +def process_sd15_info(sd15_file, model_info, parent, model_type, refetch_old): + """ Creates/Processes [model_name].json """ + + # sd v1.5 model info file + sd_data = {} + + util.printD(f"Write model SD webui info to file: {sd15_file}") + + sd_data["description"] = parent.get("description", "") + + # I suppose notes are more for user notes, but populating it + # with potentially useful information about this particular + # version of the model is fine too, right? The user can + # always replace these if they're unneeded or add to them + version_info = model_info.get("description", None) + if version_info is not None: + sd_data["notes"] = version_info + + # AFAIK civitai model versions are currently: + # SD 1.4, SD 1.5, SD 2.0, SD 2.0 786, SD 2.1, SD 2.1 786 + # SD 2.1 Unclip, SDXL 0.9, SDXL 1.0, and Other. + # Conveniently, the 4th character is all we need for webui. + # + # INFO: On Civitai, all models list base model/"sd version". + # The SD WebUI interface only displays them for Lora/Lycoris. + # I'm populating the field anyways in hopes it eventually gets + # added. + base_model = model_info.get("baseModel", None) + sd_version = 'Unknown' + if base_model: + version = base_model[3] + + sd_version = { + "1": 'SD1', + "2": 'SD2', + "L": 'SDXL', + }.get(version, 'Unknown') + + sd_data["sd version"] = sd_version + + # INFO: On Civitai, all non-checkpoint models can have trained words. + # The SD WebUI interface only displays them for Lora/Lycoris. + # I'm populating the field anyways in hopes it eventually gets + # added. + # + # "trained words" usage is inconsistent among model authors. + # Some use each entry as an individual activator, while others + # use them as entire prompts + activator = model_info.get("trainedWords", []) + if (activator and activator[0]): + if "," in activator[0]: + # assume trainedWords is a prompt list + + # webui does not support newlines in activator text + # so this is the best hinting I can give the user at the + # moment that these are mutually-exclusive prompts. + sd_data["activation text"] = " || ".join(activator) + else: + # assume trainedWords are single keywords + sd_data["activation text"] = ", ".join(activator) + + # Sadly, Civitai does not provide default weight information, + # So 0 disables this functionality on webui's end and uses + # the user's global setting + if model_type in ["lora", "lycoris"]: + sd_data["preferred weight"] = 0 + + sd_data["extensions"] = util.create_extension_block(sd_data.get("extensions", None)) + + if refetch_old: + if verify_overwrite_eligibility(sd15_file, sd_data): + write_info(sd_data, sd15_file, "webui") + else: + write_info(sd_data, sd15_file, "webui") + + +def load_model_info(path): + """ Opens a JSON file and loads its JSON """ + model_info = None + with open(os.path.realpath(path), 'r') as json_file: + try: + model_info = json.load(json_file) + except ValueError: + util.printD(f"Selected file is not json: {path}") + return None + + return model_info + + +def get_potential_model_preview_files(model_path): + """ + Find existing preview images, if any. + + Extensions from `find_preview` method in webui `modules/ui_extra_networks.py` + gif added in webui commit c602471b85d270e8c36707817d9bad92b0ff991e + + return: preview_files + """ + preview_exts = ["png", "jpg", "jpeg", "webp", "gif"] + preview_files = [] + + base, _ = os.path.splitext(model_path) + + for ext in preview_exts: + preview_files.append(f"{base}.preview.{ext}") + + return preview_files + + +def get_model_files_from_model_path(model_path): + """ return: list of paths """ + + base, _ = os.path.splitext(model_path) + + info_file, sd15_file = get_model_info_paths(model_path) + user_preview_path = f"{base}.png" + + paths = [model_path, info_file, sd15_file, user_preview_path] + preview_paths = get_potential_model_preview_files(model_path) + + paths = paths + preview_paths + + return [path for path in paths if os.path.isfile(path)] + + +def get_model_names_by_type(model_type:str) -> list: + """ + get model file names by model type + parameter: model_type - string + return: model name list + """ + + if model_type == "lora" and folders['lycoris']: + model_folders = [folders[model_type], folders['lycoris']] + else: + model_folders = [folders[model_type]] + + # get information from filter + # only get those model names don't have a civitai model info file + model_names = [] + for model_folder in model_folders: + for root, _, files in os.walk(model_folder, followlinks=True): + for filename in files: + item = os.path.join(root, filename) + # check extension + _, ext = os.path.splitext(item) + if ext in EXTS: + # find a model + model_names.append(filename) + + return model_names + + +# return 2 values: (model_root, model_path) +def get_model_path_by_type_and_name(model_type:str, model_name:str) -> str: + """ return: model_path:str matching model_name and model_type """ + util.printD("Run get_model_path_by_type_and_name") + if not model_name: + util.printD("model name can not be empty") + return None + + model_folders = [folders.get(model_type, None)] + + if model_folders[0] is None: + util.printD(f"unknown model_type: {model_type}") + return None + + if model_type == "lora" and folders['lycoris']: + model_folders.append(folders['lycoris']) + + # model could be in subfolder, need to walk. + model_path = util.find_file_in_folders(model_folders, model_name) + + msg = util.indented_msg(f""" + Got following info: + {model_path=} + """) + util.printD(msg) + + # May return `None` + return model_path + + +# get model path by model type and search_term +# parameter: model_type, search_term +# return: model_path +def get_model_path_by_search_term(model_type, search_term): + """ + Gets a model path based on the webui search term. + + return: model_path + """ + util.printD(f"Search model of {search_term} in {model_type}") + if folders.get(model_type, None) is None: + util.printD("unknow model type: " + model_type) + return None + + # for lora: search_term = subfolderpath + model name + ext + " " + hash. + # And it always start with a / even there is no sub folder + # for ckp: search_term = subfolderpath + model name + ext + " " + hash + # for ti: search_term = subfolderpath + model name + ext + " " + hash + # for hyper: search_term = subfolderpath + model name + + # this used to be + # `model_sub_path = search_term.split()[0]` + # but it was failing on models containing spaces. + model_hash = search_term.split()[-1] + model_sub_path = search_term.replace(f" {model_hash}", "") + + if model_type == "hyper": + model_sub_path = f"{search_term}.pt" + + if model_sub_path[:1] == "/": + model_sub_path = model_sub_path[1:] + + if model_type == "lora" and folders['lycoris']: + model_folders = [folders[model_type], folders['lycoris']] + else: + model_folders = [folders[model_type]] + + for folder in model_folders: + model_folder = folder + model_path = os.path.join(model_folder, model_sub_path) + + if os.path.isfile(model_path): + break + + msg = util.indented_msg(f""" + Got following info: + {model_folder=} + {model_sub_path=} + {model_path=} + """) + util.printD(msg) + + if not os.path.isfile(model_path): + util.printD(f"Can not find model file: {model_path}") + return None + + return model_path + + +pattern = re.compile(r"\s*([^:,]+):\s*([^,]+)") + +def sd_format(data): + """ + Parse image exif data for image creation parameters. + + return parameters:dict or None + """ + + if not data: + return None + + prompt = "" + negative = "" + setting = "" + + steps_index = data.find("\nSteps:") + + if steps_index != -1: + prompt = data[:steps_index].strip() + setting = data[steps_index:].strip() + + if "Negative prompt:" in data: + prompt_index = data.find("\nNegative prompt:") + + if steps_index != -1: + negative = data[ + prompt_index + len("Negative prompt:") + 1 : steps_index + ].strip() + + else: + negative = data[ + prompt_index + len("Negative prompt:") + 1 : + ].strip() + + prompt = data[:prompt_index].strip() + + elif steps_index == -1: + prompt = data + + setting_dict = dict(re.findall(pattern, setting)) + + data = { + "prompt": prompt, + "negative": negative, + "Steps": setting_dict.get("Steps", ""), + "Sampler": setting_dict.get("Sampler", ""), + "CFG_scale": setting_dict.get("CFG scale", ""), + "Seed": setting_dict.get("Seed", ""), + "Size": setting_dict.get("Size", ""), + } + + return data + + +def parse_image(image_file): + """ + Read image exif for userComment entry. + return: userComment:str + """ + data = None + with Image.open(image_file) as image: + if image.format == "PNG": + # However, unlike other image formats, EXIF data is not + # guaranteed to be present in info until load() has been called. + # https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html#png + image.load() + data = image.info.get("parameters") + + elif image.format in ["JPEG", "WEBP"]: + try: + usercomment = piexif.ExifIFD.UserComment + exif = image.info.get("exif") + if not exif: + return None + jpegexif = piexif.load(exif) or {} + data = piexif.helper.UserComment.load( + jpegexif.get("Exif", {}).get(usercomment, None) + ) + + except (ValueError, TypeError): + util.printD("Failed to parse image exif.") + return None + + return data + + +def get_remote_image_info(img_src): + """ + Download a remote image and parse out its creation parameters + + return parameters:dict or None + """ + # anti-DDOS protection + util.delay(0.2) + + success, response = downloader.request_get(img_src) + + if not success: + return None + + image_file = response.raw + try: + data = parse_image(image_file) + + except OSError: #, UnidentifiedImageError + util.printD("Failed to open image.") + return None + + if not data: + return None + + sd_data = sd_format(data) + return sd_data + + +def update_civitai_info_image_meta(filename): + """ + Read model metadata and update missing image creation parameters, + if available. + """ + need_update = False + data = {} + + if not os.path.isfile(filename): + return + + with open(filename, 'r') as model_json: + data = json.load(model_json) + + for image in data.get('images', []): + metadata = image.get('meta', None) + if not metadata and metadata != {}: + url = image.get("url", "") + if not url: + continue + + util.printD(f"{filename} missing generation info for {url}. Processing {url}.") + + image_data = get_remote_image_info(url) + if not image_data: + util.printD(f"Failed to find generation info on remote image at {url}.") + + # "mark" image so additional runs will skip it. + image["meta"] = {} + need_update = True + continue + + util.printD( + "The following information will be added to " + f"{filename} for {url}:\n{image_data}" + ) + metadata = image_data + image["meta"] = metadata + + need_update = True + + if need_update: + with open(filename, 'w') as info_file: + json.dump(data, info_file, indent=4) + + +def scan_civitai_info_image_meta(): + """ Search for *.civitai.info files """ + util.printD("Start Scan_civitai_info_image_meta") + output = "" + count = 0 + + directories = [y for x, y in folders.items() if os.path.isdir(y)] + util.printD(f"{directories=}") + for directory in directories: + for root, _, files in os.walk(directory): + for filename in files: + if filename.endswith('.civitai.info'): + update_civitai_info_image_meta(os.path.join(root, filename)) + count = count + 1 + + output = f"Done. Scanned {count} files." + util.printD(output) + return output diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/model_action_civitai.py b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/model_action_civitai.py new file mode 100644 index 0000000000000000000000000000000000000000..4d68342febba9b984ffbaecdf22fe1965ba9849f --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/model_action_civitai.py @@ -0,0 +1,711 @@ +""" -*- coding: UTF-8 -*- +handle msg between js and python side +""" +import os +import time +import re +from modules import sd_models +from . import util +from . import model +from . import civitai +from . import downloader +from . import templates + + +def get_metadata_skeleton(): + """ + Used to generate at least something when model is not on civitai. + """ + return { + "id": "", + "modelId": "", + "name": "", + "trainedWords": [], + "baseModel": "Unknown", + "description": "", + "model": { + "name": "", + "type": "", + "nsfw": "", + "poi": "" + }, + "files": [ + { + "name": "", + "sizeKB": 0, + "type": "Model", + "hashes": { + "AutoV2": "", + "SHA256": "" + } + } + ], + "downloadUrl": "" + } + + +def scan_single_model(filename, root, model_type, refetch_old, delay): + """ + Gets model info for a model by feeding its sha256 hash into civitai's api + + return: success:bool + """ + + # check ext + item = os.path.join(root, filename) + _, ext = os.path.splitext(item) + if ext not in model.EXTS: + return False + + # find a model, get info file + info_file, sd15_file = model.get_model_info_paths(item) + + # check info file + if model.metadata_needed(info_file, sd15_file, refetch_old): + util.printD(f"Creating model info for: {filename}") + # get model's sha256 + sha256_hash = util.gen_file_sha256(item) + + if not sha256_hash: + output = f"failed generating SHA256 for model: {filename}" + util.printD(output) + return False + + # use this sha256 to get model info from civitai + model_info = civitai.get_model_info_by_hash(sha256_hash) + + if (model_info == {}) and not model_info.get("id", None): + model_info = dummy_model_info(item, sha256_hash, model_type) + + model.process_model_info(item, model_info, model_type, refetch_old=refetch_old) + + # delay before next request, to prevent being treated as a DDoS attack + util.printD(f"delay: {delay} second") + time.sleep(delay) + + else: + util.printD(f"Model metadata not needed for {filename}") + + return True + + +def scan_model(scan_model_types, max_size_preview, nsfw_preview_threshold, refetch_old): + """ Scan model to generate SHA256, then use this SHA256 to get model info from civitai + return output msg + """ + + delay = 0.2 + + util.printD("Start scan_model") + output = "" + + # check model types + if not scan_model_types: + output = "Model Types is None, can not scan." + util.printD(output) + yield output + return + + model_types = [] + + # check type if it is a string + if isinstance(scan_model_types, str): + model_types.append(scan_model_types) + else: + model_types = scan_model_types + + count = 0 + + for model_type, model_folder in model.folders.items(): + if model_type not in model_types: + continue + + util.printD(f"Scanning path: {model_folder}") + for root, _, files in os.walk(model_folder, followlinks=True): + for filename in files: + success = scan_single_model(filename, root, model_type, refetch_old, delay) + + if not success: + continue + + # set model_count + count = count + 1 + + # check preview image + filepath = os.path.join(root, filename) + + # webui-visible progress bar + for result in civitai.get_preview_image_by_model_path( + filepath, + max_size_preview, + nsfw_preview_threshold + ): + yield result + + # this previously had an image count, but it always matched the model count. + output = f"Done. Scanned {count} models." + + util.printD(output) + + yield output + + +def dummy_model_info(path, sha256_hash, model_type): + """ + Fills model metadata with information we can get locally. + """ + if not sha256_hash: + return {} + + model_info = get_metadata_skeleton() + + autov2 = sha256_hash[:10] + filename = os.path.basename(path) + filesize = os.path.getsize(path) // 1024 + + model_metadata = model_info["model"] + file_metadata = model_info["files"][0] + + model_metadata["name"] = filename + model_metadata["type"] = model_type + + file_metadata["name"] = filename + file_metadata["sizeKB"] = filesize + file_metadata["hashes"]["SHA256"] = sha256_hash + file_metadata["hashes"]["AutoV2"] = autov2 + + # We can't get data on the model from civitai, but some models + # do store their training data. + trained_words = model_info["trainedWords"] + + try: + file_metadata = sd_models.read_metadata_from_safetensors(path) + except AssertionError: + # model is not a safetensors file. This is fine, + # it just doesn't have metadata we can read + pass + + tag_frequency = file_metadata.get("ss_tag_frequency", {}) + + for tag in tag_frequency.keys(): + word = re.sub(r"^\d+_", "", tag) + trained_words.append(word) + + return model_info + + +def get_model_info_by_input( + model_type, model_name, model_url_or_id, max_size_preview, nsfw_preview_threshold +): + """ + Get model info by model type, name and url + output is log info to display on markdown component + """ + output = "" + + # parse model id + model_id = civitai.get_model_id_from_url(model_url_or_id) + if not model_id: + output = f"failed to parse model id from url: {model_url_or_id}" + util.printD(output) + return output + + # get model file path + # model could be in subfolder + model_path = model.get_model_path_by_type_and_name(model_type, model_name) + + if model_path is None: + output = "Could not get Model Path" + util.printD(output) + return output + + # get model info + #we call it model_info, but in civitai, it is actually version info + model_info = civitai.get_version_info_by_model_id(model_id) + + model.process_model_info(model_path, model_info, model_type) + + # check preview image + webui-visible progress bar + for result in civitai.get_preview_image_by_model_path( + model_path, + max_size_preview, + nsfw_preview_threshold + ): + yield result + + yield output + + +def build_article_from_version(version): + """ + Builds the HTML for displaying new model versions to the user. + + return: html:str + """ + ( + model_path, model_id, model_name, new_version_id, + new_version_name, description, download_url, + img_url, model_type + ) = version + + thumbnail = "" + if img_url: + thumbnail = templates.thumbnail.substitute( + img_url=img_url, + ) + + if download_url: + # replace "\" to "/" in model_path for windows + download_model_path = model_path.replace('\\', '\\\\') + + download_section = templates.download.substitute( + new_version_id=new_version_id, + new_version_name=new_version_name, + model_path=download_model_path, + model_type=model_type, + download_url=download_url + ) + + else: + download_section = templates.no_download.substitute( + new_version_name=new_version_name, + ) + + description_section = "" + if description: + description_section = templates.description.substitute( + description=util.safe_html(download_section), + ) + + article = templates.article.substitute( + url=f'{civitai.URLS["modelPage"]}{model_id}', + thumbnail=thumbnail, + download=download_section, + description=description_section, + model_name=model_name, + model_path=model_path + ) + + return article + + +def check_models_new_version_to_md(model_types:list) -> str: + """ + check models' new version and output to UI as html doc + return: html:str + """ + new_versions = civitai.check_models_new_version_by_model_types(model_types, 0.2) + + if not new_versions: + util.printD("Done: no new versions found.") + return "No models have new versions" + + articles = [] + count = 0 + for count, new_version in enumerate(new_versions): + article = build_article_from_version(new_version) + articles.append(article) + + output = f"Found new versions for following models:
{''.join(articles)}
" + + if count != 1: + util.printD(f"Done. Found {count} models that have new versions. Check UI for detail") + else: + util.printD(f"Done. Found {count} model that has a new version. Check UI for detail.") + + return output + + +def get_model_info_by_url(model_url_or_id:str) -> tuple: + """ + Retrieves model information necessary to populate HTML + with Model Name, Model Type, valid saving directories, + and available model versions. + + return: tuple or None + """ + util.printD(f"Getting model info by: {model_url_or_id}") + + # parse model id + model_id = civitai.get_model_id_from_url(model_url_or_id) + if not model_id: + util.printD("Could not parse model id from url or id") + return None + + model_info = civitai.get_model_info_by_id(model_id) + if model_info is None: + util.printD("Connection to Civitai API service failed. Wait a while and try again") + return None + + if not model_info: + util.printD("Failed to get model info from url or id") + return None + + # parse model type, model name, subfolder, version from this model info + # get model type + civitai_model_type = model_info.get("type", None) + if civitai_model_type not in civitai.MODEL_TYPES: + util.printD(f"This model type is not supported: {civitai_model_type}") + return None + + model_type = civitai.MODEL_TYPES[civitai_model_type] + + # get model type + model_name = model_info.get("name", None) + if model_name is None: + util.printD("model name is Empty") + model_name = "" + + # get version lists + model_versions = model_info.get("modelVersions", None) + if model_versions is None: + util.printD("modelVersions is Empty") + return None + + version_strs = [] + for version in model_versions: + # version name can not be used as id + # version id is not readable + # so , we use name_id as version string + version_str = f'{version["name"]}_{version["id"]}' + version_strs.append(version_str) + + # get folder by model type + folder = model.folders[model_type] + + # get subfolders + subfolders = ["/"] + util.get_subfolders(folder) + + msg = util.indented_msg(f""" + Got following info for downloading: + {model_name=} + {model_type=} + {version_strs=} + {subfolders=} + """) + util.printD(msg) + + return (model_info, model_name, model_type, subfolders, version_strs) + + +def get_ver_info_by_ver_str(version_str:str, model_info:dict) -> dict: + """ + get version info by version string + + return: version_info:dict + """ + + if not (version_str and model_info): + output = util.indented_msg( + f""" + Missing Parameter: + {model_info=} + {version_str=} + """ + ) + util.printD(output) + return None + + # get version list + model_versions = model_info.get("modelVersions", None) + if model_versions is None: + util.printD("modelVersions is Empty") + return None + + # find version by version_str + version = None + for ver in model_versions: + # version name can not be used as id + # version id is not readable + # so , we use name_id as version string + ver_str = f'{ver["name"]}_{ver["id"]}' + if ver_str == version_str: + # find version + version = ver + break + + if not (version and ("id" in version)): + util.printD(f"can not find version or id by version string: {version_str}") + return None + + return version + + +def get_id_and_dl_url_by_version_str(version_str:str, model_info:dict) -> tuple: + """ + get download url from model info by version string + return - (version_id, download_url) + """ + if not (version_str and model_info): + output = util.indented_msg(f""" + Missing Parameter: + {model_info=} + {version_str=} + """) + util.printD(output) + return (False, output) + + # get version list + model_versions = model_info.get("modelVersions", None) + if model_versions is None: + util.printD("modelVersions is Empty") + return (False, output) + + # find version by version_str + version = None + for ver in model_versions: + # version name can not be used as id + # version id is not readable + # so , we use name_id as version string + ver_str = f'{ver["name"]}_{ver["id"]}' + if ver_str == version_str: + # find version + version = ver + break + + version_id = None + download_url = None + if version: + download_url = version.get("downloadUrl", None) + version_id = version.get("id", None) + + if None in [version, version_id, download_url]: + output = util.indented_msg(f""" + Invalid Version Information: + {version=} + {version_id=} + {download_url=} + """) + util.printD(output) + return (False, output) + + util.printD(f"Get Download Url: {download_url}") + + return (version_id, download_url) + + +def download_all(model_folder, ver_info, headers, duplicate): + """ + get all download url from files info + some model versions have multiple files + """ + + version_id = ver_info["id"] + + download_urls = [] + + for file_info in ver_info.get("files", {}): + download_url = file_info.get("downloadUrl", None) + if download_url is not None: + download_urls.append(download_url) + + if len(download_urls) == 0: + download_url = ver_info.get("downloadUrl", None) + if download_url is not None: + download_urls.append(download_url) + + # check if this model already exists + result = civitai.search_local_model_info_by_version_id(model_folder, version_id) + if result: + output = "This model version already exists" + util.printD(output) + yield (False, output) + + # download + success = False + output = "" + filepath = "" + file_candidate = "" + total = len(download_urls) + errors = [] + errors_count = 0 + + for count, url in enumerate(download_urls): + + snippet = "" + if errors_count > 0: + snippet = f"| {errors_count}/{total} models failed" + + # webui visible progress bar + for result in downloader.dl_file( + url, folder=model_folder, duplicate=duplicate, + headers=headers + ): + if not isinstance(result, str): + success, output = result + break + + yield f"{result} | {count}/{total} models {snippet}" + + if not success: + errors.append(downloader.error(url, output)) + errors_count += 1 + continue + + file_candidate = output + + if url == ver_info["downloadUrl"]: + filepath = file_candidate + + if not filepath: + filepath = file_candidate + + additional = None + if errors_count > 0: + additional = "\n\n".join(errors) + + if errors_count == total: + yield (False, additional) + return + + yield (True, filepath, additional) + + +def download_one(model_folder, ver_info, headers, duplicate): + """ + only download one file + get download url + """ + + download_url = ver_info["downloadUrl"] + + output = "" + if not download_url: + output = "Failed to find a download url" + util.printD(output) + yield (False, output) + + # download + success = False + for result in downloader.dl_file( + download_url, folder=model_folder, duplicate=duplicate, + headers=headers + ): + if not isinstance(result, str): + success, output = result + break + + yield result + + if not success: + downloader.error(download_url, output) + yield (False, output) + + yield (True, output) + + +def dl_model_by_input( + model_info:dict, + model_type:str, + subfolder_str:str, + version_str:str, + dl_all_bool:bool, + max_size_preview:bool, + nsfw_preview_threshold:bool, + duplicate:str +) -> str: + """ download model from civitai by input + output to markdown log + """ + if not (model_info and model_type and subfolder_str and version_str): + output = util.indented_msg(f""" + Missing Required Parameter in dl_model_by_input. Parameters given: + {model_type=}* + {subfolder_str=}* + {version_str=}* + {dl_all_bool=} + {max_size_preview=} + {nsfw_preview_threshold=} + {duplicate=} + """) + + # Keep model info away from util.indented_msg + # which can screw with complex strings + output = f"{output}\n {model_info=}*\n * Required" + util.printD(output) + yield output + return + + # get model root folder + if model_type not in model.folders: + output = f"Unsupported model type: {model_type}" + util.printD(output) + yield output + return + + folder = "" + subfolder = "" + output = "" + version_info = None + + model_root_folder = model.folders[model_type] + + # get subfolder + if subfolder_str in ["/", "\\"]: + subfolder = "" + elif subfolder_str[:1] in ["/", "\\"]: + subfolder = subfolder_str[1:] + else: + subfolder = subfolder_str + + # get model folder for downloading + folder = os.path.join(model_root_folder, subfolder) + if not os.path.isdir(folder): + output = f"Model folder is not a dir: {folder}" + util.printD(output) + yield output + return + + # get version info + ver_info = get_ver_info_by_ver_str(version_str, model_info) + if not ver_info: + output = "Failed to get version info, check console log for detail" + util.printD(output) + yield output + return + + success = False + downloader_fn = download_one + if dl_all_bool: + downloader_fn = download_all + + headers = { + "content-type": "application/json" + } + api_key = util.get_opts("ch_civiai_api_key") + if api_key: + headers["Authorization"] = f"Bearer {api_key}" + + additional = None + for result in downloader_fn(folder, ver_info, headers, duplicate): + if not isinstance(result, str): + if len(result) > 2: + success, output, additional = result + else: + success, output = result + + break + + yield result + + if not success: + yield output + return + + # get version info + version_info = civitai.get_version_info_by_version_id(ver_info["id"]) + model.process_model_info(output, version_info, model_type) + + # then, get preview image + webui-visible progress + for result in civitai.get_preview_image_by_model_path( + output, + max_size_preview, + nsfw_preview_threshold + ): + yield f"Downloading model preview:\n{result}" + + output = f"Done. Model downloaded to: {output}" + if additional: + output = f"{output}. Additionally, the following failures occurred: \n{additional}" + util.printD(output) + yield output diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/msg_handler.py b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/msg_handler.py new file mode 100644 index 0000000000000000000000000000000000000000..f07c76751218805aa06ec65bd64a838c26be6ca3 --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/msg_handler.py @@ -0,0 +1,74 @@ +""" -*- coding: UTF-8 -*- +handle msg between js and python side +""" +import json +from . import util + +# action list +JS_ACTIONS = ( + "open_url", + "add_trigger_words", + "use_preview_prompt", + "dl_model_new_version", + "rename_card", + "remove_card" +) + +PY_ACTIONS = ( + "open_url", + "rename_card", + "remove_card" +) + + +def parse_js_msg(msg): + """ + handle request from javascript + parameter: msg - msg from js as string in a hidden textbox + return: dict for result + """ + util.printD("Start parse js msg") + msg_dict = json.loads(msg) + + # in case client side run JSON.stringify twice + if isinstance(msg_dict, str): + msg_dict = json.loads(msg_dict) + + action = msg_dict.get("action", "") + if not action: + util.printD("No action from js request") + return None + + if action not in JS_ACTIONS: + util.printD(f"Unknown action: {action}") + return None + + util.printD("End parse js msg") + + return msg_dict + + +def build_py_msg(action:str, content:dict): + """ + build python side msg for sending to js + parameter: content dict + return: msg as string, to fill into a hidden textbox + """ + util.printD("Start build_msg") + if not (content and action and action in PY_ACTIONS): + util.indented_msg( + f""" + Could not run action on content: + {action=} + {content=} + """ + ) + return None + + msg = { + "action" : action, + "content": content + } + + util.printD("End build_msg") + return json.dumps(msg) diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/templates.py b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/templates.py new file mode 100644 index 0000000000000000000000000000000000000000..b63fb589ff4bd018bc4fdc7dbccbf97259e06189 --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/templates.py @@ -0,0 +1,46 @@ +""" +HTML templates +""" + +from string import Template +from . import util + +article = Template(util.dedent(""" +
+ $thumbnail +
+ File: $model_path +
+ $download +
+ Model: $model_name +
+
+""").strip()) + +thumbnail = Template(util.dedent(""" + +""").strip()) + +description = Template(util.dedent(""" +
+ $description +
+
+""").strip()) + +# add js function to download new version into SD webui by python +# in embed HTML, onclick= will also follow a ", never a ', +# so have to write it as following +download = Template(util.dedent(""" +
+ New Version: $new_version_name + [Download into SD] +
+""").strip()) + +no_download = Template(util.dedent(""" +
+ New Version: $new_version_name +
+""").strip()) diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/util.py b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/util.py new file mode 100644 index 0000000000000000000000000000000000000000..6b090ae5e1a1cb413e59c51003d659f4cd3d60c8 --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/util.py @@ -0,0 +1,445 @@ +""" -*- coding: UTF-8 -*- +Utility functions for Stable Diffusion Civitai Helper +""" +from __future__ import annotations +import os +import io +import re +import hashlib +import textwrap +import time +import subprocess +import gradio as gr +from modules.shared import opts +from modules import hashes +import launch +from packaging.version import parse as parse_version + +# used to append extension information to JSON/INFO files +SHORT_NAME = "sd_civitai_helper" + +# current version of the exension +VERSION = "1.7.7" + +# Civitai INFO files below this version will regenerated +COMPAT_VERSION_CIVITAI = "1.7.2" + +# SD webui model info JSON below this version will be regenerated +COMPAT_VERSION_SDWEBUI = "1.7.4" + +DEFAULT_HEADERS = { + "User-Agent": ( + "Mozilla/5.0 (iPad; CPU OS 12_2 like Mac OS X) " + "AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148" + ) +} + +PROXIES = { + "http": None, + "https": None, +} + +REQUEST_TIMEOUT = 300 # 5 minutes +REQUEST_RETRIES = 5 + +_MINUTE = 60 +_HOUR = _MINUTE * 60 +_DAY = _HOUR * 24 + + +# print for debugging +def printD(msg:any) -> str: + """ Print a message to stderr """ + print(f"Civitai Helper: {msg}") + + +def append_default_headers(headers:dict) -> dict: + """ Append extension default values to customized headers where missing """ + + for key, val in DEFAULT_HEADERS.items(): + if key not in headers: + headers[key] = val + return headers + + +def indented_msg(msg:str) -> str: + """ + Clean up an indented message in the format of + [header] + var1=var1 + var2=var2 + var3=var3 + + and print the results in the format of: + + Civitai Helper: [header] + var1: var1 + var2: var2 + var3: var3 + + return: msg:str + """ + + msg_parts = textwrap.dedent( + msg + ).strip().split('\n') + msg = [msg_parts.pop(0)] + for part in msg_parts: + part = ": ".join(part.split("=")) + msg.append(f" {part}") + msg = "\n".join(msg) + + return msg + + +def delay(seconds:float) -> None: + """ delay before next request, mostly to prevent to be treated as DDoS """ + printD(f"delay: {seconds} second") + time.sleep(seconds) + + +def is_stale(timestamp:float) -> bool: + """ Returns if a timestamp was more than a day ago. """ + cur_time = ch_time() + elapsed = cur_time - timestamp + + if elapsed > _DAY: + return True + + return False + + +def info(msg:str) -> None: + """ Display an info smessage on the client DOM """ + gr.Info(msg) + + +def warning(msg:str) -> None: + """ Display a warning message on the client DOM """ + gr.Warning(msg) + + +def error(msg:str) -> None: + """ Display an error message on the client DOM """ + gr.Error(msg) + + +def ch_time() -> int: + """ Unix timestamp """ + return int(time.time()) + + +def dedent(text:str) -> str: + """ alias for textwrap.dedent """ + return textwrap.dedent(text) + + +def read_chunks(file, size=io.DEFAULT_BUFFER_SIZE) -> bytes: + """ Yield pieces of data from a file-like object until EOF. """ + while True: + chunk = file.read(size) + if not chunk: + break + yield chunk + + +def get_name(model_path:str) -> str: + """ return: lora/{model_name}:str """ + + _, filename = os.path.split(model_path) + model_name, _ = os.path.splitext(filename) + return f"lora/{model_name}" + + +def get_opts(key): + """ return: option value """ + return opts.data.get(key, None) + + +def gen_file_sha256(filename:str) -> str: + """ return a sha256 hash for a file """ + + if get_opts("ch_use_sdwebui_sha256"): + printD("Using SD Webui SHA256") + name = get_name(filename) + return hashes.sha256(filename, name, use_addnet_hash=False) + + # pip-style sha256 hash generation + printD("Use Memory Optimized SHA256") + blocksize=1 << 20 + sha256_hash = hashlib.sha256() + length = 0 + with open(os.path.realpath(filename), 'rb') as read_file: + for block in read_chunks(read_file, size=blocksize): + length += len(block) + sha256_hash.update(block) + + hash_value = sha256_hash.hexdigest() + printD(f"sha256: {hash_value}") + printD(f"length: {length}") + return hash_value + + +def get_subfolders(folder:str) -> list[str]: + """ return: list of subfolders """ + printD(f"Get subfolder for: {folder}") + if not folder: + printD("folder can not be None") + return [] + + if not os.path.isdir(folder): + printD("path is not a folder") + return [] + + prefix_len = len(folder) + full_dirs_searched = [] + subfolders = [] + for root, dirs, _ in os.walk(folder, followlinks=True): + if root == folder: + continue + + # Prevent following recursive symlinks + follow = [] + for directory in dirs: + full_dir_path = os.path.join(root, directory) + try: + canonical_dir = os.path.realpath(full_dir_path, strict=True) + if canonical_dir not in full_dirs_searched: + full_dirs_searched.append(canonical_dir) + follow.append(directory) + + except OSError: + printD(f"Symlink loop: {directory}") + continue + + # Get subfolder path + subfolder = root[prefix_len:] + subfolders.append(subfolder) + + # Update dirs parameter to prevent following recursive symlinks + dirs[:] = follow + + return subfolders + + +def find_file_in_folders(folders:list, filename:str) -> str: + """ + Searches a directory for a filename, + + return: filename:str or None + """ + for folder in folders: + for root, _, files in os.walk(folder, followlinks=True): + if filename in files: + # found file + return os.path.join(root, filename) + + return None + + +# get relative path +def get_relative_path(item_path:str, parent_path:str) -> str: + """ + Gets a relative path from an absolute path and its parent_path + item path must start with parent_path + return: relative_path:str + """ + + if not (item_path and parent_path): + return "" + + if not item_path.startswith(parent_path): + # return absolute path + return item_path + + relative = item_path[len(parent_path):] + if relative[:1] == "/" or relative[:1] == "\\": + relative = relative[1:] + + # printD(f"relative: {relative}") + return relative + + +# Allowed HTML tags +whitelist = re.compile(r"]*>") + +# Allowed HTML attributes +attrs = re.compile(r"""(?:href|src|target)=['"]?[^\s'"]*['"]?""") + +def safe_html_replace(match:match) -> str: + """ Given a block of text, returns that block with most HTML removed + and unneeded attributes pruned. + """ + tag = None + attr = None + close = False + + match = whitelist.match(match.group(0)) + if match is not None: + html_elem = match.group(0) + tag = match.group(1) + close = html_elem[1] == "/" + if (tag in ["a", "img"]) and not close: + sub_match = attrs.findall(html_elem) + if sub_match is not None: + attr = " ".join(sub_match) + + if close: + return f"" + + return f"<{tag} {attr}>" if attr else f"<{tag}>" + + return "" + +def safe_html(html:str) -> str: + """ whitelist only HTML I"m comfortable displaying in webui """ + + return re.sub("<[^<]+?>", safe_html_replace, html) + + +def trim_html(html:str) -> str: + """ Remove any HTML for a given string and, if needed, replace it with + a comparable plain-text alternative. + """ + + def sub_tag(match): + tag = match.group(1) + if tag == "/p": + return "\n\n" + if tag == "br": + return "\n" + if tag == "li": + return "* " + if tag in ["code", "/code"]: + return "`" + return '' + + def sub_escaped(match): + escaped = match.group(1) + unescaped = { + "gt": ">", + "lt": "<", + "quot": '"', + "amp": "&" + } + return unescaped.get(escaped, "") + + # non-breaking space. Useless unstyled content + html = html.replace("\u00a0", "") + + # remove non-whitelisted HTML tags, + # replace whitelisted tags with text-equivalents + html = re.sub(r"<(/?[a-zA-Z]+)(?:[^>]+)?>", sub_tag, html) + + # Replace HTML-escaped characters with displayables. + html = re.sub(r"\&(gt|lt|quot|amp)\;", sub_escaped, html) + + # Because we encapsulate the description in HTML comment, + # We have to prevent those comments from being cancelled. + html.replace("-->", "→") + + # https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/13241 + return f"" + + +def newer_version(ver1:str, ver2:str, allow_equal=False) -> bool: + """ Returns ver1 > ver2 + if allow_equal, returns ver1 >= ver2 + """ + if allow_equal: + return parse_version(ver1) >= parse_version(ver2) + + return parse_version(ver1) > parse_version(ver2) + + +def metadata_version(metadata:dict) -> str | bool: + """ Attempts retrieve the extension version used to create + to create the object block + """ + try: + return metadata["extensions"][SHORT_NAME]["version"] + except KeyError: + return False + + +def create_extension_block(data=None) -> dict: + """ Creates or edits an extensions block for usage in JSON files + created or edited by this extension. + + Adds the current version of this extension to the extensions block + """ + + cur_time = ch_time() + + block = { + SHORT_NAME: { + "version": VERSION, + "last_update": cur_time + } + } + + if not data: + return block + + if not data.get(SHORT_NAME, False): + data[SHORT_NAME] = block[SHORT_NAME] + return data + + data[SHORT_NAME]["version"] = VERSION + data[SHORT_NAME]["last_update"] = cur_time + + return data + + +def webui_version() -> str: + ''' Gets the current webui version using webui's launch tools + + The version is expected to be in the format `v1.6.0-128-g792589fd`, + tho all that is explicitly required is `vX`. + + returns the version in the form 'X.Y.Z' + ''' + version = None + try: + tag = launch.git_tag() + match = re.match(r"v([\d.]+)", tag) + if match: + version = match.group(1) + else: + # XXX assume a modern SD Webui version if one cannot be found. + version = "1.6.0" + + except AttributeError: + try: + return subprocess.check_output( + ["git", "describe", "--tags"], + shell=False, + encoding='utf8' + ).strip() + + except subprocess.SubprocessError: + try: + changelog_md = os.path.join( + os.path.dirname(os.path.dirname(__file__)), + "CHANGELOG.md" + ) + with open(changelog_md, "r", encoding="utf-8") as file: + line = next((line.strip() for line in file if line.strip()), "") + line = line.replace("## ", "") + version = line + + except OSError: + version = "1.6.0" + + return version + + +filename_re = re.compile(r"[^A-Za-z\d\s\^\-\+_.\(\)\[\]]") +def bash_filename(filename:str) -> str: + """ + Bashes a filename with a large fish until I'm comfortable using it. + + Keeps a limited set of valid characters, but does not account for + reserved names like COM. + """ + return re.sub(filename_re, "", filename) diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/civitai_helper.py b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/civitai_helper.py new file mode 100644 index 0000000000000000000000000000000000000000..81fa8904b6e6e01a35e9281564fd7deb8703d74b --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/scripts/civitai_helper.py @@ -0,0 +1,575 @@ +""" -*- coding: UTF-8 -*- +This extension can help you manage your models from civitai. + It can download preview, add trigger words, open model page and use the prompt from preview image +repo: https://github.com/butaixianran/ +""" + +import os +import gradio as gr +import modules +from modules import scripts +from modules import shared +from modules import script_callbacks +from scripts.ch_lib import model +from scripts.ch_lib import js_action_civitai +from scripts.ch_lib import model_action_civitai +from scripts.ch_lib import civitai +from scripts.ch_lib import util + +# init +# root path +ROOT_PATH = os.getcwd() + +# extension path +EXTENSION_PATH = scripts.basedir() + +# default hidden values for civitai helper buttons +BUTTONS = { + "replace_preview_button": False, + "open_url_button": False, + "add_trigger_words_button": util.newer_version(util.webui_version(), '1.5.0', allow_equal=True), + "add_preview_prompt_button": False, + "rename_model_button": False, + "remove_model_button": False, +} + +model.get_custom_model_folder() + +def on_ui_tabs(): + # init + # init_py_msg = { + # # relative extension path + # "EXTENSION_PATH": util.get_relative_path(EXTENSION_PATH, ROOT_PATH), + # } + # init_py_msg_str = json.dumps(init_py_msg) + + # set proxy + proxy = util.get_opts("ch_proxy") + if proxy: + util.printD(f"Set Proxy: {proxy}") + util.PROXIES["http"] = proxy + util.PROXIES["https"] = proxy + + # get prompt textarea + # check modules/ui.py, search for txt2img_paste_fields + # Negative prompt is the second element + txt2img_prompt = modules.ui.txt2img_paste_fields[0][0] + txt2img_neg_prompt = modules.ui.txt2img_paste_fields[1][0] + img2img_prompt = modules.ui.img2img_paste_fields[0][0] + img2img_neg_prompt = modules.ui.img2img_paste_fields[1][0] + + # ====Event's function==== + def get_model_names_by_input(model_type, empty_info_only): + names = civitai.get_model_names_by_input(model_type, empty_info_only) + return model_name_drop.update(choices=names) + + def get_model_info_by_url(url, subfolder): + request = model_action_civitai.get_model_info_by_url(url) + + model_info = {} + model_name = "" + model_type = "" + subfolders = [] + version_strs = [] + if request: + model_info, model_name, model_type, subfolders, version_strs = request + + if subfolder == "" or subfolder not in subfolders: + subfolder = "/" + + return [ + model_info, model_name, model_type, + dl_subfolder_drop.update( + choices=subfolders, + value=subfolder + ), + dl_version_drop.update( + choices=version_strs, + value=version_strs[0] + ) + ] + + # ====UI==== + with gr.Blocks(analytics_enabled=False) as civitai_helper: + # with gr.Blocks(css=".block.padded {padding: 10px !important}") as civitai_helper: + + # init + max_size_preview = util.get_opts("ch_max_size_preview") + nsfw_preview_threshold = util.get_opts("ch_nsfw_preview_threshold") + proxy = util.get_opts("ch_proxy") + + model_types = list(model.folders.keys()) + no_info_model_names = civitai.get_model_names_by_input("ckp", False) + + # session data + dl_model_info = gr.State({}) + + with gr.Box(elem_classes="ch_box"): + with gr.Column(): + gr.Markdown("### Scan Models for Civitai") + with gr.Row(): + with gr.Column(): + max_size_preview_ckb = gr.Checkbox( + label="Download Max Size Preview", + value=max_size_preview, + elem_id="ch_max_size_preview_ckb" + ) + with gr.Column(): + nsfw_preview_threshold_drop = gr.Dropdown( + label="Block NSFW Level Above", + choices=civitai.NSFW_LEVELS[1:], + value=nsfw_preview_threshold, + elem_id="ch_nsfw_preview_threshold_drop" + ) + with gr.Column(): + refetch_old_ckb = gr.Checkbox( + label="Replace Old Metadata Formats*", + value=False, + elem_id="ch_refetch_old_ckb" + ) + gr.HTML(""" +
* [wiki]
Do not use this option if you have made changes with the metadata editor without backing up your data!! + """) + with gr.Column(): + scan_model_types_drop = gr.CheckboxGroup( + choices=model_types, + label="Model Types", + value=model_types + ) + + # with gr.Row(): + scan_model_civitai_btn = gr.Button( + value="Scan", + variant="primary", + elem_id="ch_scan_model_civitai_btn" + ) + + scan_civitai_info_image_meta_btn = gr.Button( + value="Update image generation information (Experimental)", + variant="primary", + elem_id="ch_Scan_civitai_info_image_meta_btn" + ) + + # with gr.Row(): + scan_model_log_md = gr.Markdown( + value="Scanning takes time, just wait. Check console log for detail", + elem_id="ch_scan_model_log_md" + ) + + + with gr.Box(elem_classes="ch_box"): + with gr.Column(): + gr.Markdown("### Get Model Info from Civitai by URL") + gr.Markdown("Use this when scanning can not find a local model on civitai") + with gr.Row(): + model_type_drop = gr.Dropdown( + choices=model_types, + label="Model Type", + value="ckp", + multiselect=False + ) + empty_info_only_ckb = gr.Checkbox( + label="Only Show Models have no Info", + value=False, + elem_id="ch_empty_info_only_ckb", + elem_classes="ch_vpadding" + ) + model_name_drop = gr.Dropdown( + choices=no_info_model_names, + label="Model", + value="ckp", + multiselect=False + ) + + model_url_or_id_txtbox = gr.Textbox( + label="Civitai URL", + lines=1, + value="" + ) + get_civitai_model_info_by_id_btn = gr.Button( + value="Get Model Info from Civitai", + variant="primary" + ) + get_model_by_id_log_md = gr.Markdown("") + + with gr.Box(elem_classes="ch_box"): + with gr.Column(): + gr.Markdown("### Download Model") + with gr.Row(): + dl_model_url_or_id_txtbox = gr.Textbox( + label="Civitai URL", + lines=1, + value="" + ) + dl_model_info_btn = gr.Button( + value="1. Get Model Info by Civitai Url", + variant="primary" + ) + + gr.Markdown(value="2. Pick Subfolder and Model Version") + with gr.Row(): + dl_model_name_txtbox = gr.Textbox( + label="Model Name", + interactive=False, + lines=1, + value="" + ) + dl_model_type_txtbox = gr.Textbox( + label="Model Type", + interactive=False, + lines=1, + value="" + ) + dl_subfolder_drop = gr.Dropdown( + choices=[], + label="Sub-folder", + value="", + interactive=True, + multiselect=False + ) + dl_version_drop = gr.Dropdown( + choices=[], + label="Model Version", + value="", + interactive=True, + multiselect=False + ) + dl_duplicate_drop = gr.Dropdown( + choices=["Skip", "Overwrite", "Rename New"], + label="Duplicate File Behavior", + value="Skip", + interactive=True, + multiselect=False + ) + dl_all_ckb = gr.Checkbox( + label="Download All files", + value=False, + elem_id="ch_dl_all_ckb", + elem_classes="ch_vpadding" + ) + + dl_civitai_model_by_id_btn = gr.Button( + value="3. Download Model", + variant="primary" + ) + dl_log_md = gr.Markdown( + value="Check Console log for Downloading Status" + ) + + with gr.Box(elem_classes="ch_box"): + with gr.Column(): + gr.Markdown("### Check models' new version") + with gr.Row(): + model_types_ckbg = gr.CheckboxGroup( + choices=model_types, + label="Model Types", + value=[ + "ti", "hyper", "ckp", "lora", "lycoris" + ] + ) + check_models_new_version_btn = gr.Button( + value="Check New Version from Civitai", + variant="primary" + ) + + check_models_new_version_log_md = gr.HTML( + "It takes time, just wait. Check console log for detail" + ) + + # ====Footer==== + gr.HTML(f"
{util.SHORT_NAME} version: {util.VERSION}
") + + # ====hidden component for js, not in any tab==== + js_msg_txtbox = gr.Textbox( + label="Request Msg From Js", + visible=False, + lines=1, + value="", + elem_id="ch_js_msg_txtbox" + ) + py_msg_txtbox = gr.Textbox( + label="Response Msg From Python", + visible=False, + lines=1, + value="", + elem_id="ch_py_msg_txtbox" + ) + + js_open_url_btn = gr.Button( + value="Open Model Url", + visible=False, + elem_id="ch_js_open_url_btn" + ) + js_add_trigger_words_btn = gr.Button( + value="Add Trigger Words", + visible=False, + elem_id="ch_js_add_trigger_words_btn" + ) + js_use_preview_prompt_btn = gr.Button( + value="Use Prompt from Preview Image", + visible=False, + elem_id="ch_js_use_preview_prompt_btn" + ) + js_dl_model_new_version_btn = gr.Button( + value="Download Model's new version", + visible=False, + elem_id="ch_js_dl_model_new_version_btn" + ) + js_rename_card_btn = gr.Button( + value="Rename Card", + visible=False, + elem_id="ch_js_rename_card_btn" + ) + js_remove_card_btn = gr.Button( + value="Remove Card", + visible=False, + elem_id="ch_js_remove_card_btn" + ) + + # ====events==== + # Scan Models for Civitai + scan_model_civitai_btn.click( + model_action_civitai.scan_model, + inputs=[ + scan_model_types_drop, max_size_preview_ckb, + nsfw_preview_threshold_drop, refetch_old_ckb + ], + outputs=scan_model_log_md + ) + + scan_civitai_info_image_meta_btn.click( + model.scan_civitai_info_image_meta, + outputs=scan_model_log_md + ) + + # Get Civitai Model Info by Model Page URL + model_type_drop.change( + get_model_names_by_input, + inputs=[ + model_type_drop, empty_info_only_ckb + ], + outputs=model_name_drop + ) + empty_info_only_ckb.change( + get_model_names_by_input, + inputs=[ + model_type_drop, empty_info_only_ckb + ], + outputs=model_name_drop + ) + + get_civitai_model_info_by_id_btn.click( + model_action_civitai.get_model_info_by_input, + inputs=[ + model_type_drop, model_name_drop, + model_url_or_id_txtbox, max_size_preview_ckb, + nsfw_preview_threshold_drop + ], + outputs=get_model_by_id_log_md + ) + + # Download Model + dl_model_info_btn.click( + get_model_info_by_url, + inputs=[ + dl_model_url_or_id_txtbox, dl_subfolder_drop + ], + outputs=[ + dl_model_info, dl_model_name_txtbox, + dl_model_type_txtbox, dl_subfolder_drop, + dl_version_drop + ] + ) + dl_civitai_model_by_id_btn.click( + model_action_civitai.dl_model_by_input, + inputs=[ + dl_model_info, dl_model_type_txtbox, + dl_subfolder_drop, dl_version_drop, + dl_all_ckb, max_size_preview_ckb, + nsfw_preview_threshold_drop, dl_duplicate_drop + ], + outputs=dl_log_md + ) + + # Check models' new version + check_models_new_version_btn.click( + model_action_civitai.check_models_new_version_to_md, + inputs=model_types_ckbg, + outputs=check_models_new_version_log_md + ) + + # js action + js_open_url_btn.click( + js_action_civitai.open_model_url, + inputs=[js_msg_txtbox], + outputs=py_msg_txtbox + ) + js_add_trigger_words_btn.click( + js_action_civitai.add_trigger_words, + inputs=[js_msg_txtbox], + outputs=[ + txt2img_prompt, img2img_prompt + ] + ) + js_use_preview_prompt_btn.click( + js_action_civitai.use_preview_image_prompt, + inputs=[js_msg_txtbox], + outputs=[ + txt2img_prompt, txt2img_neg_prompt, + img2img_prompt, img2img_neg_prompt + ] + ) + js_dl_model_new_version_btn.click( + js_action_civitai.dl_model_new_version, + inputs=[ + js_msg_txtbox, max_size_preview_ckb, + nsfw_preview_threshold_drop + ], + outputs=dl_log_md + ) + js_rename_card_btn.click( + js_action_civitai.rename_model_by_path, + inputs=[js_msg_txtbox], + outputs=py_msg_txtbox + ) + js_remove_card_btn.click( + js_action_civitai.remove_model_by_path, + inputs=[js_msg_txtbox], + outputs=py_msg_txtbox + ) + + # the third parameter is the element id on html, with a "tab_" as prefix + return ((civitai_helper, "Civitai Helper", "civitai_helper"),) + + +def on_ui_settings(): + section = ('civitai_helper', "Civitai Helper") + shared.opts.add_option( + "ch_civiai_api_key", + shared.OptionInfo( + "", + ( + "API key for authenticating with Civitai. " + "This is required to download some models. " + "See Wiki for more details." + ), + gr.Textbox, + {"interactive": True}, + section=section + ).link( + "Wiki", + "https://github.com/zixaphir/Stable-Diffusion-Webui-Civitai-Helper/wiki/Civitai-API-Key" + ) + ) + shared.opts.add_option( + "ch_open_url_with_js", + shared.OptionInfo( + True, + ( + "Open model Url on the user's client side, rather than server side. " + "If you are running WebUI locally, disabling this may open URLs in your " + "default internet browser if it is different than the one you are running " + "WebUI in" + ), + gr.Checkbox, + {"interactive": True}, + section=section + ) + ) + shared.opts.add_option( + "ch_hide_buttons", + shared.OptionInfo( + [x for x, y in BUTTONS.items() if y], + "Hide checked Civitai Helper buttons on model cards", + gr.CheckboxGroup, + {"choices": list(BUTTONS)}, + section=section + ) + ) + shared.opts.add_option( + "ch_always_display", + shared.OptionInfo( + False, + "Always Display Buttons on model cards", + gr.Checkbox, + {"interactive": True}, + section=section + ) + ) + shared.opts.add_option( + "ch_show_btn_on_thumb", + shared.OptionInfo( + True, + "Show Button On Thumb Mode in SD webui versions before 1.5.0", + gr.Checkbox, + {"interactive": True}, + section=section + ) + ) + shared.opts.add_option( + "ch_max_size_preview", + shared.OptionInfo( + True, + "Download Max Size Preview", + gr.Checkbox, + {"interactive": True}, + section=section + ) + ) + shared.opts.add_option( + "ch_nsfw_preview_threshold", + shared.OptionInfo( + civitai.NSFW_LEVELS[-1], # Allow all + util.dedent( + """ + Block NSFW images of a certain threshold and higher. + Civitai marks all images for NSFW models as also being NSFW. + These ratings do not seem to be explicitly defined on Civitai's + end, but "Soft" seems to be suggestive, with NSFW elements but + not explicit nudity, "Mature" seems to include nudity but not + always, and "X" seems to be explicitly adult content. + """ + ).strip().replace("\n", " "), + gr.Dropdown, + { + "choices": civitai.NSFW_LEVELS[1:], + "interactive": True + }, + section=section + ) + ) + shared.opts.add_option( + "ch_dl_webui_metadata", + shared.OptionInfo( + True, + "Also add data for WebUI metadata editor", + gr.Checkbox, + {"interactive": True}, + section=section) + ) + shared.opts.add_option( + "ch_proxy", + shared.OptionInfo( + "", + "Proxy to use for fetching models and model data. Format: http://127.0.0.1:port", + gr.Textbox, + {"interactive": True}, + section=section) + ) + shared.opts.add_option( + "ch_use_sdwebui_sha256", + shared.OptionInfo( + False, + ( + "Use SD webui's built-in hashing functions for model hashes. " + "If SD webui was launced with `--no-hashing`, hashing will fail, " + "but this provides a hash cache, which should make repeat model " + "scanning faster." + ), + gr.Checkbox, + {"interactive": True}, + section=section) + ) + +script_callbacks.on_ui_settings(on_ui_settings) +script_callbacks.on_ui_tabs(on_ui_tabs) diff --git a/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/style.css b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/style.css new file mode 100644 index 0000000000000000000000000000000000000000..6a0e62687c8ccfd8b74c1514cf1c9ab50db7bbcd --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/Stable-Diffusion-Webui-Civitai-Helper/style.css @@ -0,0 +1,73 @@ +blockquote ul { + list-style:disc; + margin:4px 40px; +} + +blockquote ol { + list-style:decimal; + margin:4px 40px; +} + +.block.padded.ch_box { + padding: 10px !important; +} + +.block.padded.ch_vpadding { + padding: 10px 0 !important; +} + +.card-button { + margin: 0px 2px; + padding: 2px; + font-size: 150%; +} + +/* SD WebUI has not line height limit for model names. + * This can result in the model name filling the entire card, + * hiding our buttons. This is honestly a WebUI issue more + * than an issue on this end, but we can fix it anyways, so + * why not? + */ +.extra-network-cards .card .name:not(:hover) { + display: block; + overflow: hidden; + max-height: 3em; +} + +.extra-network-thumbs .card-button { + font-size: 100%; + display: inline; + position: static; + background-image: none; + background: rgba(0, 0, 0, 0.8); +} + +/* Lobe theme */ +.actions .additional > a { + display: none; +} + +/* + * Gross, ugly, and I hate it, but the only way to override an + * !important is with a more granular !important. + */ +.acss-culdhq .extra-networks .actions .additional > ul a.card-button { + margin: 0px 2px !important; + padding: 2px !important; + font-size: 150% !important; + +} + +/* Work around a display bug in Lobe that causes lowers opacity to 0.2 */ +#txt2img_extra_tabs [style*="_cards_html"] .pending, +#img2img_extra_tabs [style*="_cards_html"] .pending { + opacity: 1; +} + +/* Work around a display bug in Lobe that causes an infinite counter */ +#txt2img_extra_tabs [id*="_cards_html"] .progress-text , +#img2img_extra_tabs [id*="_cards_html"] .progress-text { + display: none; +} + +/* end of Lobe theme */ diff --git a/AiAF/Backup-SD-Extensions-Folder/a1111-sd-webui-tagcomplete/.gitignore b/AiAF/Backup-SD-Extensions-Folder/a1111-sd-webui-tagcomplete/.gitignore new file mode 100644 index 0000000000000000000000000000000000000000..e9e3707f8a1eaacfbd6b478874868b91f3f97587 --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/a1111-sd-webui-tagcomplete/.gitignore @@ -0,0 +1,2 @@ +tags/temp/ +__pycache__/ diff --git a/AiAF/Backup-SD-Extensions-Folder/a1111-sd-webui-tagcomplete/LICENSE b/AiAF/Backup-SD-Extensions-Folder/a1111-sd-webui-tagcomplete/LICENSE new file mode 100644 index 0000000000000000000000000000000000000000..72a5479833a0bc73e8b2e4b39467e7ccfc8bdc0c --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/a1111-sd-webui-tagcomplete/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2022 Dominik Reh + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/AiAF/Backup-SD-Extensions-Folder/a1111-sd-webui-tagcomplete/README.md b/AiAF/Backup-SD-Extensions-Folder/a1111-sd-webui-tagcomplete/README.md new file mode 100644 index 0000000000000000000000000000000000000000..1deb335014d1ef1f94e3f5d7f48d14dd41c14430 --- /dev/null +++ b/AiAF/Backup-SD-Extensions-Folder/a1111-sd-webui-tagcomplete/README.md @@ -0,0 +1,598 @@ +![tag_autocomplete_light](https://user-images.githubusercontent.com/34448969/208306863-90bbd663-2cb4-47f1-a7fe-7b662a7b95e2.png) + +
+ +# SD WebUI Tag Autocomplete +## English • [简体中文](./README_ZH.md) • [日本語](./README_JA.md) + +Booru style tag autocompletion for the AUTOMATIC1111 Stable Diffusion WebUI + +[![Github Release][release-shield]][release-url] +[![stargazers][stargazers-shield]][stargazers-url] +[![contributors][contributors-shield]][contributors-url] +[![forks][forks-shield]][forks-url] +[![issues][issues-shield]][issues-url] + +[Changelog][release-url] • +[Known Issues](#%EF%B8%8F-common-problems--known-issues) • +[Report Bug][issues-url] • +[Request Feature][issues-url] +
+
+ +# 📄 Description + +Tag Autocomplete is an extension for the popular [AUTOMATIC1111 web UI](https://github.com/AUTOMATIC1111/stable-diffusion-webui) for Stable Diffusion. + +It displays autocompletion hints for recognized tags from "image booru" boards such as Danbooru, which are primarily used for browsing Anime-style illustrations. +Since some Stable Diffusion models were trained using this information, for example [Waifu Diffusion](https://github.com/harubaru/waifu-diffusion) and many of the NAI-descendant models or merges, using exact tags in prompts can often improve composition and consistency. + +You can install it using the inbuilt available extensions list, clone the files manually as described [below](#-installation), or use a pre-packaged version from [Releases](https://github.com/DominikDoom/a1111-sd-webui-tagcomplete/releases). + +
+ +# ✨ Features +- 🚀 Instant completion hints while typing (under normal circumstances) +- ⌨️ Keyboard navigation +- 🌒 Dark & Light mode support +- 🛠️ Many [settings](#%EF%B8%8F-settings) and customizability +- 🌍 [Translation support](#translations) for tags, with optional live preview for the full prompt + - **Note:** Translation files are provided by the community, see [here](#list-of-translations) for a list of translations I know of. + +Tag autocomplete supports built-in completion for: +- 🏷️ **Danbooru & e621 tags** (Top 100k by post count, as of November 2022) +- ✳️ [**Wildcards**](#wildcards) +- ➕ [**Extra network**](#extra-networks-embeddings-hypernets-lora-) filenames, including + - Textual Inversion embeddings [(jump to readme section)] + - Hypernetworks + - LoRA + - LyCORIS / LoHA +- 🪄 [**Chants**](#chants) (custom format for longer prompt presets) +- 🏷️ "[**Extra file**](#extra-file)", one set of customizable extra tags + + +Additionally, some support for other third party extensions exists: +
+Click to expand + +- [Image Browser][image-browser-url] - Filename & EXIF keyword search +- [Multidiffusion Upscaler][multidiffusion-url] - Regional Prompts +- [Dataset Tag Editor][tag-editor-url] - Caption, Interrogate Result, Edit Tags & Edit Caption +- [WD 1.4 Tagger][wd-tagger-url] - Additional & Excluded tags +- [Umi AI][umi-url] - Completion for YAML wildcards +
+
+ +# 🖼️ Screenshots & Demo videos +
+Click to expand +Basic usage (with keyboard navigation): + +https://user-images.githubusercontent.com/34448969/200128020-10d9a8b2-cea6-4e3f-bcd2-8c40c8c73233.mp4 + +Wildcard script support: + +https://user-images.githubusercontent.com/34448969/200128031-22dd7c33-71d1-464f-ae36-5f6c8fd49df0.mp4 + +Extra Network preview support: + +https://github.com/DominikDoom/a1111-sd-webui-tagcomplete/assets/34448969/3c0cad84-fb5f-436d-b05a-28db35860d13 + +Dark and Light mode supported, including tag colors: + +![results_dark](https://user-images.githubusercontent.com/34448969/200128214-3b6f21b4-9dda-4acf-820e-5df0285c30d6.png) +![results_light](https://user-images.githubusercontent.com/34448969/200128217-bfac8b60-6673-447b-90fd-dc6326f1618c.png) +
+
+ +# 📦 Installation +## Using the built-in extension list +1. Open the `Extensions` tab +2. Open the `Available` sub-tab +3. Click **Load from** +4. Find **Booru tag autocompletion** in the list + - The extension was one of the first available, so selecting "oldest first" will show it high up in the list. + - Alternatively, use CRTL + F to search for the text on the page +5. Click **Install** on the right side + +![Load from index](https://github.com/DominikDoom/a1111-sd-webui-tagcomplete/assets/34448969/b9b860c1-2e77-41b1-aa5c-a44e252f1a40) +![Order by oldest](https://user-images.githubusercontent.com/34448969/223537231-48e982b8-0920-48c5-87e5-8c81ebbb5fe3.png) +![Install](https://user-images.githubusercontent.com/34448969/223537336-5c02ccb1-233d-4e0d-9e73-d1b889252c49.png) + + +## Manual clone +```bash +git clone "https://github.com/DominikDoom/a1111-sd-webui-tagcomplete.git" extensions/tag-autocomplete +``` +(The second argument specifies the name of the folder, you can choose whatever you like). + +
+ +# ❇️ Additional completion support +## Wildcards +Autocompletion also works with wildcard files used by https://github.com/AUTOMATIC1111/stable-diffusion-webui-wildcards or other similar scripts/extensions. +Completion is triggered by typing `__` (double underscore). It will first show a list of your wildcard files, and upon choosing one, the replacement options inside that file. +This enables you to either insert categories to be replaced by the script, or directly choose one and use wildcards as a sort of categorized custom tag system. + +![Wildcard files](https://user-images.githubusercontent.com/34448969/223534518-8488c2e1-d9e5-4870-844f-adbf3bfb1eee.png) +![Wildcard replacements](https://user-images.githubusercontent.com/34448969/223534534-69597907-59de-4ba8-ae83-b01386570124.png) + + +Wildcards are searched for in every extension folder, as well as the `scripts/wildcards` folder to support legacy versions. This means that you can combine wildcards from multiple extensions. Nested folders are also supported if you have grouped your wildcards in that way. + +## Extra networks (Embeddings, Hypernets, LoRA, ...) +Completion for these types is triggered by typing `<`. By default it will show them all mixed together, but further filtering can be done in the following way: +- `