aa926620ca6701673dec0f931c22897dc1c41bb94a3cec6e3b01f666d53a734a

microsoftexcel-supermerger/elemental_ja.md

@@ -0,0 +1,119 @@
+# Elemental Merge
+- 階層マージを越えた階層マージです
+
+階層マージでは25の階層ごとにマージ比率を変えることができますが、階層もまた複数の要素で構成されており、要素ごとに比率を変えることも原理的には可能です。可能ですが、要素の数は600以上にもなり人の手で扱えるのかは疑問でしたが実装してみました。いきなり要素ごとのマージは推奨されません。階層マージにおいて解決不可能な問題が生じたときに最終調節手段として使うことをおすすめします。  
+次の画像はOUT05層の要素を変えた結果です。左端はマージ無し。2番目はOUT05層すべて(つまりは普通の階層マージ),以降が要素マージです。下表のとおり、attn2などの中にはさらに複数の要素が含まれます。
+![](https://raw.githubusercontent.com/hako-mikan/sd-webui-supermerger/images/sample1.jpg)
+
+## 使い方
+要素マージは通常マージ、階層マージ時どちらの場合でも有効で、最後に計算されるために、階層マージで指定した値は上書きされることに注意してください。
+
+Elemental Mergeで設定します。ここにテキストが設定されていると自動的に適応されるので注意して下さい。各要素は下表のとおりですが、各要素のフルネームを入力する必要はありません。  
+ちゃんと効果が現れるかどうかはprint changeチェックを有効にすることで確認できます。このチェックを有効にするとマージ時にコマンドプロンプト画面に適用された要素が表示されます。  
+部分一致で指定が可能です。
+### 書式
+階層:要素:比率,階層:要素:比率,...  
+または  
+階層:要素:比率  
+階層:要素:比率  
+階層:要素:比率  
+
+カンマまたは改行で区切ることで複数の指定が可能です。カンマと改行は混在しても問題ありません。
+階層は大文字でBASE,IN00-M00-OUT11まで指定でます。空欄にするとすべての階層に適用されます。スペースで区切ることで複数の階層を指定できます。
+要素も同様でスペースで区切ることで複数の要素を指定できます。  
+部分一致で判断するので、例えば「attn」と入力するとattn1,attn2両方が変化します。「attn2」の場合はattn2のみ。さらに細かく指定したい場合は「attn2.to_out」などと入力します。  
+
+OUT03 OUT04 OUT05:attn2 attn1.to_out:0.5  
+
+と入力すると、OUT03,OUT04,OUT05層のattn2が含まれる要素及びattn1.to_outの比率が0.5になります。
+要素の欄を空欄にすると指定階層のすべての要素が変わり、階層マージと同じ効果になります。
+指定が重複する場合、後に入力された方が優先されます。 
+
+OUT06:attn:0.5,OUT06:attn2.to_k:0.2  
+
+と入力した場合、OUT06層のattn2.to_k以外のattnは0.5,attn2.to_kのみ0.2となります。  
+
+最初にNOTと入力することで効果範囲を反転させることができます。
+これは階層・要素別に設定できます。  
+
+NOT OUT04:attn:1  
+
+と入力するとOUT04層以外の層のattnに比率1が設定されます。  
+
+OUT05:NOT attn proj:0.2  
+
+とすると、OUT05層のattnとproj以外の層が0.2になります。
+
+## XY plot
+elemental用のXY plotを複数用意しています。入力例はsample.txtにあります。  
+#### elemental
+複数の要素マージについてXY plotを作成します。要素同士は空行で区切ってください。
+トップ画像はsample.txtのsample1を実行した結果です。
+
+#### pinpoint element
+特定の要素について値を変えてXY plotを作成します。pinpoint Blocksと同じことを要素で行います。反対の軸にはalphaを指定してください。要素同士は改行またはカンマで区切ります。  
+以下の画像はsample.txtのsample3を実行した結果です。
+![](https://raw.githubusercontent.com/hako-mikan/sd-webui-supermerger/images/sample3.jpg)
+
+#### effective elenemtal checker
+各要素の影響度を差分として出力します。オプションでanime gif、csvファイルを出力できます。gif.csvファイルはoutputフォルダにModelAとModelBから作られるフォルダ下に作成されるdiffフォルダに作成されます。ファイル名が重複する場合名前を変えて保存しますが、増えてくるとややこしいのでdiffフォルダを適当な名前に変えることをおすすめします。  
+改行またはカンマで区切ります。反対の軸はalphaを使用し、単一の値を入力してください。これは要素の効果を見るのにも有効ですが、要素を指定しないことで階層の効果を見ることも可能なので、そちらの使い方をする場合が多いかもしれません。  
+以下��画像はsample.txtのsample5を実行した結果です。
+![](https://raw.githubusercontent.com/hako-mikan/sd-webui-supermerger/images/sample5-1.jpg)
+![](https://raw.githubusercontent.com/hako-mikan/sd-webui-supermerger/images/sample5-2.jpg)
+### 要素一覧
+基本的にはattnが顔や服装の情報を担っているようです。特にIN07,OUT03,OUT04,OUT05層の影響度が強いようです。階層によって影響度が異なることが多いので複数の層の同じ要素を同時に変化させることは意味が無いように思えます。
+nullと書かれた場所には要素が存在しません。
+
+||IN00|IN01|IN02|IN03|IN04|IN05|IN06|IN07|IN08|IN09|IN10|IN11|M00|M00|OUT00|OUT01|OUT02|OUT03|OUT04|OUT05|OUT06|OUT07|OUT08|OUT09|OUT10|OUT11
+|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|
+op.bias|null|null|null||null|null||null|null||null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null
+op.weight|null|null|null||null|null||null|null||null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null
+emb_layers.1.bias|null|||null|||null|||null|null|||||||||||||||
+emb_layers.1.weight|null|||null|||null|||null|null|||||||||||||||
+in_layers.0.bias|null|||null|||null|||null|null|||||||||||||||
+in_layers.0.weight|null|||null|||null|||null|null|||||||||||||||
+in_layers.2.bias|null|||null|||null|||null|null|||||||||||||||
+in_layers.2.weight|null|||null|||null|||null|null|||||||||||||||
+out_layers.0.bias|null|||null|||null|||null|null|||||||||||||||
+out_layers.0.weight|null|||null|||null|||null|null|||||||||||||||
+out_layers.3.bias|null|||null|||null|||null|null|||||||||||||||
+out_layers.3.weight|null|||null|||null|||null|null|||||||||||||||
+skip_connection.bias|null|null|null|null||null|null||null|null|null|null|null|null||||||||||||
+skip_connection.weight|null|null|null|null||null|null||null|null|null|null|null|null||||||||||||
+norm.bias|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+norm.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+proj_in.bias|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+proj_in.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+proj_out.bias|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+proj_out.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.attn1.to_k.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.attn1.to_out.0.bias|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.attn1.to_out.0.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.attn1.to_q.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.attn1.to_v.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.attn2.to_k.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.attn2.to_out.0.bias|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.attn2.to_out.0.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.attn2.to_q.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.attn2.to_v.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.ff.net.0.proj.bias|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.ff.net.0.proj.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.ff.net.2.bias|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.ff.net.2.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.norm1.bias|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.norm1.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.norm2.bias|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.norm2.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.norm3.bias|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+transformer_blocks.0.norm3.weight|null|||null|||null|||null|null|null||null|null|null|null|||||||||
+conv.bias|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null||null|null||null|null||null|null|null
+conv.weight|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null||null|null||null|null||null|null|null
+0.bias||null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|
+0.weight||null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|
+2.bias|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|
+2.weight|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|
+time_embed.0.weight||null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|
+time_embed.0.bias||null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|
+time_embed.2.weight||null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|
+time_embed.2.bias||null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|null|

microsoftexcel-supermerger/install.py

@@ -0,0 +1,7 @@
+import launch
+
+if not launch.is_installed("sklearn"):
+    launch.run_pip("install scikit-learn", "scikit-learn")
+
+if not launch.is_installed("diffusers"):
+    launch.run_pip("install diffusers", "diffusers")

microsoftexcel-supermerger/sample.txt

@@ -0,0 +1,95 @@
+eamples of XY plot
+***************************************************************
+for elemental
+Each value is separated by a blank line/空行で区切ります
+Element merge values are comma or newline delimited/要素マージの値はカンマまたは改行区切りです
+Commas and newlines can also be mixed/カンマと改行の混在も可能です
+You can insert no elemental by inserting two blank lines at the beginning/最初に空行をふたつ入れるとelementalのない状態を挿入できます
+
+**sample1*******************************************************
+
+
+OUT05::1
+
+OUT05:layers:1
+
+OUT05:attn1:1
+
+OUT05:attn2:1
+
+OUT05:ff.net:1
+**sample2*******************************************************
+
+
+IN07:attn1:0.5,IN07:attn2:0.2
+OUT03:NOT attn1.to_k.weight:0.5
+OUT03:attn1.to_k.weight:0.3
+
+OUT04:NOT ff.net:0.5
+OUT04:attn1.to_k.weight:0.3
+OUT04:attn1.to_out.0.weight:0.3
+
+OUT05:NOT ff.net:0.5
+OUT05:attn1.to_k.weight:0.3
+OUT05:attn1.to_out.0.weight:0.3
+***************************************************************
+***************************************************************
+for pinpoint element
+Each value is separated by comma or a blank line/カンマまたは改行で区切ります
+do not enter ratio/ratioは入力しません
+**sample3******************************************************
+IN07:,IN07:layers,IN07:attn1,IN07:attn2,IN07:ff.net
+**sample4******************************************************
+OUT04:NOT attn2.to_q
+OUT04:attn1.to_k.weight
+OUT04:ff.net
+OUT04:attn
+***************************************************************
+***************************************************************
+for effective elemental cheker
+
+Examine the effect of each block/各階層の影響度を調べる
+Output results can be split by inserting "|"/途中「|」を挿入することで出力結果を分割できます。
+**sample5*************************************************************
+IN00:,IN01:,IN02:,IN03:,IN04:,IN05:,IN06:,IN07:,IN08:,IN09:,IN10:,IN11:,M00:|OUT00:,OUT01:,OUT02:,OUT03:,OUT04:,OUT05:,OUT06:,OUT07:,OUT08:,OUT09:,OUT10:,OUT11:
+
+Examine the effect of all elements in the IN01/IN01層のすべての要素の影響度を調べる
+Below that corresponds to IN02 or later/その下はIN02以降に対応
+**sample6*************************************************************
+IN01:emb_layers.1.bias,IN01:emb_layers.1.weight,IN01:in_layers.0.bias,IN01:in_layers.0.weight,IN01:in_layers.2.bias,IN01:in_layers.2.weight,IN01:out_layers.0.bias,IN01:out_layers.0.weight,IN01:out_layers.3.bias,IN01:out_layers.3.weight,IN01:skip_connection.bias,IN01:skip_connection.weight,IN01:norm.bias,IN01:norm.weight,IN01:proj_in.bias,IN01:proj_in.weight,IN01:proj_out.bias,IN01:proj_out.weight,IN01:transformer_blocks.0.attn1.to_k.weight,IN01:transformer_blocks.0.attn1.to_out.0.bias,IN01:transformer_blocks.0.attn1.to_out.0.weight,IN01:transformer_blocks.0.attn1.to_q.weight,IN01:transformer_blocks.0.attn1.to_v.weight,IN01:transformer_blocks.0.attn2.to_k.weight,IN01:transformer_blocks.0.attn2.to_out.0.bias,IN01:transformer_blocks.0.attn2.to_out.0.weight,IN01:transformer_blocks.0.attn2.to_q.weight,IN01:transformer_blocks.0.attn2.to_v.weight,IN01:transformer_blocks.0.ff.net.0.proj.bias,IN01:transformer_blocks.0.ff.net.0.proj.weight,IN01:transformer_blocks.0.ff.net.2.bias,IN01:transformer_blocks.0.ff.net.2.weight,IN01:transformer_blocks.0.norm1.bias,IN01:transformer_blocks.0.norm1.weight,IN01:transformer_blocks.0.norm2.bias,IN01:transformer_blocks.0.norm2.weight,IN01:transformer_blocks.0.norm3.bias,IN01:transformer_blocks.0.norm3.weight
+
+IN02:emb_layers.1.bias,IN02:emb_layers.1.weight,IN02:in_layers.0.bias,IN02:in_layers.0.weight,IN02:in_layers.2.bias,IN02:in_layers.2.weight,IN02:out_layers.0.bias,IN02:out_layers.0.weight,IN02:out_layers.3.bias,IN02:out_layers.3.weight,IN02:skip_connection.bias,IN02:skip_connection.weight,IN02:norm.bias,IN02:norm.weight,IN02:proj_in.bias,IN02:proj_in.weight,IN02:proj_out.bias,IN02:proj_out.weight,IN02:transformer_blocks.0.attn1.to_k.weight,IN02:transformer_blocks.0.attn1.to_out.0.bias,IN02:transformer_blocks.0.attn1.to_out.0.weight,IN02:transformer_blocks.0.attn1.to_q.weight,IN02:transformer_blocks.0.attn1.to_v.weight,IN02:transformer_blocks.0.attn2.to_k.weight,IN02:transformer_blocks.0.attn2.to_out.0.bias,IN02:transformer_blocks.0.attn2.to_out.0.weight,IN02:transformer_blocks.0.attn2.to_q.weight,IN02:transformer_blocks.0.attn2.to_v.weight,IN02:transformer_blocks.0.ff.net.0.proj.bias,IN02:transformer_blocks.0.ff.net.0.proj.weight,IN02:transformer_blocks.0.ff.net.2.bias,IN02:transformer_blocks.0.ff.net.2.weight,IN02:transformer_blocks.0.norm1.bias,IN02:transformer_blocks.0.norm1.weight,IN02:transformer_blocks.0.norm2.bias,IN02:transformer_blocks.0.norm2.weight,IN02:transformer_blocks.0.norm3.bias,IN02:transformer_blocks.0.norm3.weight
+
+IN00:bias,IN00:weight,IN03:op.bias,IN03:op.weight,IN06:op.bias,IN06:op.weight,IN09:op.bias,IN09:op.weight
+
+IN04:emb_layers.1.bias,IN04:emb_layers.1.weight,IN04:in_layers.0.bias,IN04:in_layers.0.weight,IN04:in_layers.2.bias,IN04:in_layers.2.weight,IN04:out_layers.0.bias,IN04:out_layers.0.weight,IN04:out_layers.3.bias,IN04:out_layers.3.weight,IN04:skip_connection.bias,IN04:skip_connection.weight,IN04:norm.bias,IN04:norm.weight,IN04:proj_in.bias,IN04:proj_in.weight,IN04:proj_out.bias,IN04:proj_out.weight,IN04:transformer_blocks.0.attn1.to_k.weight,IN04:transformer_blocks.0.attn1.to_out.0.bias,IN04:transformer_blocks.0.attn1.to_out.0.weight,IN04:transformer_blocks.0.attn1.to_q.weight,IN04:transformer_blocks.0.attn1.to_v.weight,IN04:transformer_blocks.0.attn2.to_k.weight,IN04:transformer_blocks.0.attn2.to_out.0.bias,IN04:transformer_blocks.0.attn2.to_out.0.weight,IN04:transformer_blocks.0.attn2.to_q.weight,IN04:transformer_blocks.0.attn2.to_v.weight,IN04:transformer_blocks.0.ff.net.0.proj.bias,IN04:transformer_blocks.0.ff.net.0.proj.weight,IN04:transformer_blocks.0.ff.net.2.bias,IN04:transformer_blocks.0.ff.net.2.weight,IN04:transformer_blocks.0.norm1.bias,IN04:transformer_blocks.0.norm1.weight,IN04:transformer_blocks.0.norm2.bias,IN04:transformer_blocks.0.norm2.weight,IN04:transformer_blocks.0.norm3.bias,IN04:transformer_blocks.0.norm3.weight
+
+IN05:emb_layers.1.bias,IN05:emb_layers.1.weight,IN05:in_layers.0.bias,IN05:in_layers.0.weight,IN05:in_layers.2.bias,IN05:in_layers.2.weight,IN05:out_layers.0.bias,IN05:out_layers.0.weight,IN05:out_layers.3.bias,IN05:out_layers.3.weight,IN05:skip_connection.bias,IN05:skip_connection.weight,IN05:norm.bias,IN05:norm.weight,IN05:proj_in.bias,IN05:proj_in.weight,IN05:proj_out.bias,IN05:proj_out.weight,IN05:transformer_blocks.0.attn1.to_k.weight,IN05:transformer_blocks.0.attn1.to_out.0.bias,IN05:transformer_blocks.0.attn1.to_out.0.weight,IN05:transformer_blocks.0.attn1.to_q.weight,IN05:transformer_blocks.0.attn1.to_v.weight,IN05:transformer_blocks.0.attn2.to_k.weight,IN05:transformer_blocks.0.attn2.to_out.0.bias,IN05:transformer_blocks.0.attn2.to_out.0.weight,IN05:transformer_blocks.0.attn2.to_q.weight,IN05:transformer_blocks.0.attn2.to_v.weight,IN05:transformer_blocks.0.ff.net.0.proj.bias,IN05:transformer_blocks.0.ff.net.0.proj.weight,IN05:transformer_blocks.0.ff.net.2.bias,IN05:transformer_blocks.0.ff.net.2.weight,IN05:transformer_blocks.0.norm1.bias,IN05:transformer_blocks.0.norm1.weight,IN05:transformer_blocks.0.norm2.bias,IN05:transformer_blocks.0.norm2.weight,IN05:transformer_blocks.0.norm3.bias,IN05:transformer_blocks.0.norm3.weight
+
+IN07:emb_layers.1.bias,IN07:emb_layers.1.weight,IN07:in_layers.0.bias,IN07:in_layers.0.weight,IN07:in_layers.2.bias,IN07:in_layers.2.weight,IN07:out_layers.0.bias,IN07:out_layers.0.weight,IN07:out_layers.3.bias,IN07:out_layers.3.weight,IN07:skip_connection.bias,IN07:skip_connection.weight,IN07:norm.bias,IN07:norm.weight,IN07:proj_in.bias,IN07:proj_in.weight,IN07:proj_out.bias,IN07:proj_out.weight,IN07:transformer_blocks.0.attn1.to_k.weight,IN07:transformer_blocks.0.attn1.to_out.0.bias,IN07:transformer_blocks.0.attn1.to_out.0.weight,IN07:transformer_blocks.0.attn1.to_q.weight,IN07:transformer_blocks.0.attn1.to_v.weight,IN07:transformer_blocks.0.attn2.to_k.weight,IN07:transformer_blocks.0.attn2.to_out.0.bias,IN07:transformer_blocks.0.attn2.to_out.0.weight,IN07:transformer_blocks.0.attn2.to_q.weight,IN07:transformer_blocks.0.attn2.to_v.weight,IN07:transformer_blocks.0.ff.net.0.proj.bias,IN07:transformer_blocks.0.ff.net.0.proj.weight,IN07:transformer_blocks.0.ff.net.2.bias,IN07:transformer_blocks.0.ff.net.2.weight,IN07:transformer_blocks.0.norm1.bias,IN07:transformer_blocks.0.norm1.weight,IN07:transformer_blocks.0.norm2.bias,IN07:transformer_blocks.0.norm2.weight,IN07:transformer_blocks.0.norm3.bias,IN07:transformer_blocks.0.norm3.weight
+
+IN08:emb_layers.1.bias,IN08:emb_layers.1.weight,IN08:in_layers.0.bias,IN08:in_layers.0.weight,IN08:in_layers.2.bias,IN08:in_layers.2.weight,IN08:out_layers.0.bias,IN08:out_layers.0.weight,IN08:out_layers.3.bias,IN08:out_layers.3.weight,IN08:skip_connection.bias,IN08:skip_connection.weight,IN08:norm.bias,IN08:norm.weight,IN08:proj_in.bias,IN08:proj_in.weight,IN08:proj_out.bias,IN08:proj_out.weight,IN08:transformer_blocks.0.attn1.to_k.weight,IN08:transformer_blocks.0.attn1.to_out.0.bias,IN08:transformer_blocks.0.attn1.to_out.0.weight,IN08:transformer_blocks.0.attn1.to_q.weight,IN08:transformer_blocks.0.attn1.to_v.weight,IN08:transformer_blocks.0.attn2.to_k.weight,IN08:transformer_blocks.0.attn2.to_out.0.bias,IN08:transformer_blocks.0.attn2.to_out.0.weight,IN08:transformer_blocks.0.attn2.to_q.weight,IN08:transformer_blocks.0.attn2.to_v.weight,IN08:transformer_blocks.0.ff.net.0.proj.bias,IN08:transformer_blocks.0.ff.net.0.proj.weight,IN08:transformer_blocks.0.ff.net.2.bias,IN08:transformer_blocks.0.ff.net.2.weight,IN08:transformer_blocks.0.norm1.bias,IN08:transformer_blocks.0.norm1.weight,IN08:transformer_blocks.0.norm2.bias,IN08:transformer_blocks.0.norm2.weight,IN08:transformer_blocks.0.norm3.bias,IN08:transformer_blocks.0.norm3.weight
+
+IN10:emb_layers.1.bias,IN10:emb_layers.1.weight,IN10:in_layers.0.bias,IN10:in_layers.0.weight,IN10:in_layers.2.bias,IN10:in_layers.2.weight,IN10:out_layers.0.bias,IN10:out_layers.0.weight,IN10:out_layers.3.bias,IN10:out_layers.3.weight,IN11:emb_layers.1.bias,IN11:emb_layers.1.weight,IN11:in_layers.0.bias,IN11:in_layers.0.weight,IN11:in_layers.2.bias,IN11:in_layers.2.weight,IN11:out_layers.0.bias,IN11:out_layers.0.weight,IN11:out_layers.3.bias,IN11:out_layers.3.weight
+
+M00:0.emb_layers.1.bias,M00:0.emb_layers.1.weight,M00:0.in_layers.0.bias,M00:0.in_layers.0.weight,M00:0.in_layers.2.bias,M00:0.in_layers.2.weight,M00:0.out_layers.0.bias,M00:0.out_layers.0.weight,M00:0.out_layers.3.bias,M00:0.out_layers.3.weight,M00:1.norm.bias,M00:1.norm.weight,M00:1.proj_in.bias,M00:1.proj_in.weight,M00:1.proj_out.bias,M00:1.proj_out.weight,M00:1.transformer_blocks.0.attn1.to_k.weight,M00:1.transformer_blocks.0.attn1.to_out.0.bias,M00:1.transformer_blocks.0.attn1.to_out.0.weight,M00:1.transformer_blocks.0.attn1.to_q.weight,M00:1.transformer_blocks.0.attn1.to_v.weight,M00:1.transformer_blocks.0.attn2.to_k.weight,M00:1.transformer_blocks.0.attn2.to_out.0.bias,M00:1.transformer_blocks.0.attn2.to_out.0.weight,M00:1.transformer_blocks.0.attn2.to_q.weight,M00:1.transformer_blocks.0.attn2.to_v.weight,M00:1.transformer_blocks.0.ff.net.0.proj.bias,M00:1.transformer_blocks.0.ff.net.0.proj.weight,M00:1.transformer_blocks.0.ff.net.2.bias,M00:1.transformer_blocks.0.ff.net.2.weight,M00:1.transformer_blocks.0.norm1.bias,M00:1.transformer_blocks.0.norm1.weight,M00:1.transformer_blocks.0.norm2.bias,M00:1.transformer_blocks.0.norm2.weight,M00:1.transformer_blocks.0.norm3.bias,M00:1.transformer_blocks.0.norm3.weight,M00:2.emb_layers.1.bias,M00:2.emb_layers.1.weight,M00:2.in_layers.0.bias,M00:2.in_layers.0.weight,M00:2.in_layers.2.bias,M00:2.in_layers.2.weight,M00:2.out_layers.0.bias,M00:2.out_layers.0.weight,M00:2.out_layers.3.bias,M00:2.out_layers.3.weight
+
+OUT00:emb_layers.1.bias,OUT00:emb_layers.1.weight,OUT00:in_layers.0.bias,OUT00:in_layers.0.weight,OUT00:in_layers.2.bias,OUT00:in_layers.2.weight,OUT00:out_layers.0.bias,OUT00:out_layers.0.weight,OUT00:out_layers.3.bias,OUT00:out_layers.3.weight,OUT00:skip_connection.bias,OUT00:skip_connection.weight,OUT01:emb_layers.1.bias,OUT01:emb_layers.1.weight,OUT01:in_layers.0.bias,OUT01:in_layers.0.weight,OUT01:in_layers.2.bias,OUT01:in_layers.2.weight,OUT01:out_layers.0.bias,OUT01:out_layers.0.weight,OUT01:out_layers.3.bias,OUT01:out_layers.3.weight,OUT01:skip_connection.bias,OUT01:skip_connection.weight,OUT02:emb_layers.1.bias,OUT02:emb_layers.1.weight,OUT02:in_layers.0.bias,OUT02:in_layers.0.weight,OUT02:in_layers.2.bias,OUT02:in_layers.2.weight,OUT02:out_layers.0.bias,OUT02:out_layers.0.weight,OUT02:out_layers.3.bias,OUT02:out_layers.3.weight,OUT02:skip_connection.bias,OUT02:skip_connection.weight,OUT02:conv.bias,OUT02:conv.weight
+
+OUT03:emb_layers.1.bias,OUT03:emb_layers.1.weight,OUT03:in_layers.0.bias,OUT03:in_layers.0.weight,OUT03:in_layers.2.bias,OUT03:in_layers.2.weight,OUT03:out_layers.0.bias,OUT03:out_layers.0.weight,OUT03:out_layers.3.bias,OUT03:out_layers.3.weight,OUT03:skip_connection.bias,OUT03:skip_connection.weight,OUT03:norm.bias,OUT03:norm.weight,OUT03:proj_in.bias,OUT03:proj_in.weight,OUT03:proj_out.bias,OUT03:proj_out.weight,OUT03:transformer_blocks.0.attn1.to_k.weight,OUT03:transformer_blocks.0.attn1.to_out.0.bias,OUT03:transformer_blocks.0.attn1.to_out.0.weight,OUT03:transformer_blocks.0.attn1.to_q.weight,OUT03:transformer_blocks.0.attn1.to_v.weight,OUT03:transformer_blocks.0.attn2.to_k.weight,OUT03:transformer_blocks.0.attn2.to_out.0.bias,OUT03:transformer_blocks.0.attn2.to_out.0.weight,OUT03:transformer_blocks.0.attn2.to_q.weight,OUT03:transformer_blocks.0.attn2.to_v.weight,OUT03:transformer_blocks.0.ff.net.0.proj.bias,OUT03:transformer_blocks.0.ff.net.0.proj.weight,OUT03:transformer_blocks.0.ff.net.2.bias,OUT03:transformer_blocks.0.ff.net.2.weight,OUT03:transformer_blocks.0.norm1.bias,OUT03:transformer_blocks.0.norm1.weight,OUT03:transformer_blocks.0.norm2.bias,OUT03:transformer_blocks.0.norm2.weight,OUT03:transformer_blocks.0.norm3.bias,OUT03:transformer_blocks.0.norm3.weight
+
+OUT04:emb_layers.1.bias,OUT04:emb_layers.1.weight,OUT04:in_layers.0.bias,OUT04:in_layers.0.weight,OUT04:in_layers.2.bias,OUT04:in_layers.2.weight,OUT04:out_layers.0.bias,OUT04:out_layers.0.weight,OUT04:out_layers.3.bias,OUT04:out_layers.3.weight,OUT04:skip_connection.bias,OUT04:skip_connection.weight,OUT04:norm.bias,OUT04:norm.weight,OUT04:proj_in.bias,OUT04:proj_in.weight,OUT04:proj_out.bias,OUT04:proj_out.weight,OUT04:transformer_blocks.0.attn1.to_k.weight,OUT04:transformer_blocks.0.attn1.to_out.0.bias,OUT04:transformer_blocks.0.attn1.to_out.0.weight,OUT04:transformer_blocks.0.attn1.to_q.weight,OUT04:transformer_blocks.0.attn1.to_v.weight,OUT04:transformer_blocks.0.attn2.to_k.weight,OUT04:transformer_blocks.0.attn2.to_out.0.bias,OUT04:transformer_blocks.0.attn2.to_out.0.weight,OUT04:transformer_blocks.0.attn2.to_q.weight,OUT04:transformer_blocks.0.attn2.to_v.weight,OUT04:transformer_blocks.0.ff.net.0.proj.bias,OUT04:transformer_blocks.0.ff.net.0.proj.weight,OUT04:transformer_blocks.0.ff.net.2.bias,OUT04:transformer_blocks.0.ff.net.2.weight,OUT04:transformer_blocks.0.norm1.bias,OUT04:transformer_blocks.0.norm1.weight,OUT04:transformer_blocks.0.norm2.bias,OUT04:transformer_blocks.0.norm2.weight,OUT04:transformer_blocks.0.norm3.bias,OUT04:transformer_blocks.0.norm3.weight
+
+OUT05:emb_layers.1.bias,OUT05:emb_layers.1.weight,OUT05:in_layers.0.bias,OUT05:in_layers.0.weight,OUT05:in_layers.2.bias,OUT05:in_layers.2.weight,OUT05:out_layers.0.bias,OUT05:out_layers.0.weight,OUT05:out_layers.3.bias,OUT05:out_layers.3.weight,OUT05:skip_connection.bias,OUT05:skip_connection.weight,OUT05:norm.bias,OUT05:norm.weight,OUT05:proj_in.bias,OUT05:proj_in.weight,OUT05:proj_out.bias,OUT05:proj_out.weight,OUT05:transformer_blocks.0.attn1.to_k.weight,OUT05:transformer_blocks.0.attn1.to_out.0.bias,OUT05:transformer_blocks.0.attn1.to_out.0.weight,OUT05:transformer_blocks.0.attn1.to_q.weight,OUT05:transformer_blocks.0.attn1.to_v.weight,OUT05:transformer_blocks.0.attn2.to_k.weight,OUT05:transformer_blocks.0.attn2.to_out.0.bias,OUT05:transformer_blocks.0.attn2.to_out.0.weight,OUT05:transformer_blocks.0.attn2.to_q.weight,OUT05:transformer_blocks.0.attn2.to_v.weight,OUT05:transformer_blocks.0.ff.net.0.proj.bias,OUT05:transformer_blocks.0.ff.net.0.proj.weight,OUT05:transformer_blocks.0.ff.net.2.bias,OUT05:transformer_blocks.0.ff.net.2.weight,OUT05:transformer_blocks.0.norm1.bias,OUT05:transformer_blocks.0.norm1.weight,OUT05:transformer_blocks.0.norm2.bias,OUT05:transformer_blocks.0.norm2.weight,OUT05:transformer_blocks.0.norm3.bias,OUT05:transformer_blocks.0.norm3.weight,OUT05:conv.bias,OUT05:conv.weight
+
+,OUT06:emb_layers.1.bias,OUT06:emb_layers.1.weight,OUT06:in_layers.0.bias,OUT06:in_layers.0.weight,OUT06:in_layers.2.bias,OUT06:in_layers.2.weight,OUT06:out_layers.0.bias,OUT06:out_layers.0.weight,OUT06:out_layers.3.bias,OUT06:out_layers.3.weight,OUT06:skip_connection.bias,OUT06:skip_connection.weight,OUT06:norm.bias,OUT06:norm.weight,OUT06:proj_in.bias,OUT06:proj_in.weight,OUT06:proj_out.bias,OUT06:proj_out.weight,OUT06:transformer_blocks.0.attn1.to_k.weight,OUT06:transformer_blocks.0.attn1.to_out.0.bias,OUT06:transformer_blocks.0.attn1.to_out.0.weight,OUT06:transformer_blocks.0.attn1.to_q.weight,OUT06:transformer_blocks.0.attn1.to_v.weight,OUT06:transformer_blocks.0.attn2.to_k.weight,OUT06:transformer_blocks.0.attn2.to_out.0.bias,OUT06:transformer_blocks.0.attn2.to_out.0.weight,OUT06:transformer_blocks.0.attn2.to_q.weight,OUT06:transformer_blocks.0.attn2.to_v.weight,OUT06:transformer_blocks.0.ff.net.0.proj.bias,OUT06:transformer_blocks.0.ff.net.0.proj.weight,OUT06:transformer_blocks.0.ff.net.2.bias,OUT06:transformer_blocks.0.ff.net.2.weight,OUT06:transformer_blocks.0.norm1.bias,OUT06:transformer_blocks.0.norm1.weight,OUT06:transformer_blocks.0.norm2.bias,OUT06:transformer_blocks.0.norm2.weight,OUT06:transformer_blocks.0.norm3.bias,OUT06:transformer_blocks.0.norm3.weight
+
+OUT07:emb_layers.1.bias,OUT07:emb_layers.1.weight,OUT07:in_layers.0.bias,OUT07:in_layers.0.weight,OUT07:in_layers.2.bias,OUT07:in_layers.2.weight,OUT07:out_layers.0.bias,OUT07:out_layers.0.weight,OUT07:out_layers.3.bias,OUT07:out_layers.3.weight,OUT07:skip_connection.bias,OUT07:skip_connection.weight,OUT07:norm.bias,OUT07:norm.weight,OUT07:proj_in.bias,OUT07:proj_in.weight,OUT07:proj_out.bias,OUT07:proj_out.weight,OUT07:transformer_blocks.0.attn1.to_k.weight,OUT07:transformer_blocks.0.attn1.to_out.0.bias,OUT07:transformer_blocks.0.attn1.to_out.0.weight,OUT07:transformer_blocks.0.attn1.to_q.weight,OUT07:transformer_blocks.0.attn1.to_v.weight,OUT07:transformer_blocks.0.attn2.to_k.weight,OUT07:transformer_blocks.0.attn2.to_out.0.bias,OUT07:transformer_blocks.0.attn2.to_out.0.weight,OUT07:transformer_blocks.0.attn2.to_q.weight,OUT07:transformer_blocks.0.attn2.to_v.weight,OUT07:transformer_blocks.0.ff.net.0.proj.bias,OUT07:transformer_blocks.0.ff.net.0.proj.weight,OUT07:transformer_blocks.0.ff.net.2.bias,OUT07:transformer_blocks.0.ff.net.2.weight,OUT07:transformer_blocks.0.norm1.bias,OUT07:transformer_blocks.0.norm1.weight,OUT07:transformer_blocks.0.norm2.bias,OUT07:transformer_blocks.0.norm2.weight,OUT07:transformer_blocks.0.norm3.bias,OUT07:transformer_blocks.0.norm3.weight
+
+,OUT08:emb_layers.1.bias,OUT08:emb_layers.1.weight,OUT08:in_layers.0.bias,OUT08:in_layers.0.weight,OUT08:in_layers.2.bias,OUT08:in_layers.2.weight,OUT08:out_layers.0.bias,OUT08:out_layers.0.weight,OUT08:out_layers.3.bias,OUT08:out_layers.3.weight,OUT08:skip_connection.bias,OUT08:skip_connection.weight,OUT08:norm.bias,OUT08:norm.weight,OUT08:proj_in.bias,OUT08:proj_in.weight,OUT08:proj_out.bias,OUT08:proj_out.weight,OUT08:transformer_blocks.0.attn1.to_k.weight,OUT08:transformer_blocks.0.attn1.to_out.0.bias,OUT08:transformer_blocks.0.attn1.to_out.0.weight,OUT08:transformer_blocks.0.attn1.to_q.weight,OUT08:transformer_blocks.0.attn1.to_v.weight,OUT08:transformer_blocks.0.attn2.to_k.weight,OUT08:transformer_blocks.0.attn2.to_out.0.bias,OUT08:transformer_blocks.0.attn2.to_out.0.weight,OUT08:transformer_blocks.0.attn2.to_q.weight,OUT08:transformer_blocks.0.attn2.to_v.weight,OUT08:transformer_blocks.0.ff.net.0.proj.bias,OUT08:transformer_blocks.0.ff.net.0.proj.weight,OUT08:transformer_blocks.0.ff.net.2.bias,OUT08:transformer_blocks.0.ff.net.2.weight,OUT08:transformer_blocks.0.norm1.bias,OUT08:transformer_blocks.0.norm1.weight,OUT08:transformer_blocks.0.norm2.bias,OUT08:transformer_blocks.0.norm2.weight,OUT08:transformer_blocks.0.norm3.bias,OUT08:transformer_blocks.0.norm3.weight,OUT08:conv.bias,OUT08:conv.weight
+
+OUT09:emb_layers.1.bias,OUT09:emb_layers.1.weight,OUT09:in_layers.0.bias,OUT09:in_layers.0.weight,OUT09:in_layers.2.bias,OUT09:in_layers.2.weight,OUT09:out_layers.0.bias,OUT09:out_layers.0.weight,OUT09:out_layers.3.bias,OUT09:out_layers.3.weight,OUT09:skip_connection.bias,OUT09:skip_connection.weight,OUT09:norm.bias,OUT09:norm.weight,OUT09:proj_in.bias,OUT09:proj_in.weight,OUT09:proj_out.bias,OUT09:proj_out.weight,OUT09:transformer_blocks.0.attn1.to_k.weight,OUT09:transformer_blocks.0.attn1.to_out.0.bias,OUT09:transformer_blocks.0.attn1.to_out.0.weight,OUT09:transformer_blocks.0.attn1.to_q.weight,OUT09:transformer_blocks.0.attn1.to_v.weight,OUT09:transformer_blocks.0.attn2.to_k.weight,OUT09:transformer_blocks.0.attn2.to_out.0.bias,OUT09:transformer_blocks.0.attn2.to_out.0.weight,OUT09:transformer_blocks.0.attn2.to_q.weight,OUT09:transformer_blocks.0.attn2.to_v.weight,OUT09:transformer_blocks.0.ff.net.0.proj.bias,OUT09:transformer_blocks.0.ff.net.0.proj.weight,OUT09:transformer_blocks.0.ff.net.2.bias,OUT09:transformer_blocks.0.ff.net.2.weight,OUT09:transformer_blocks.0.norm1.bias,OUT09:transformer_blocks.0.norm1.weight,OUT09:transformer_blocks.0.norm2.bias,OUT09:transformer_blocks.0.norm2.weight,OUT09:transformer_blocks.0.norm3.bias,OUT09:transformer_blocks.0.norm3.weight
+
+OUT10:emb_layers.1.bias,OUT10:emb_layers.1.weight,OUT10:in_layers.0.bias,OUT10:in_layers.0.weight,OUT10:in_layers.2.bias,OUT10:in_layers.2.weight,OUT10:out_layers.0.bias,OUT10:out_layers.0.weight,OUT10:out_layers.3.bias,OUT10:out_layers.3.weight,OUT10:skip_connection.bias,OUT10:skip_connection.weight,OUT10:norm.bias,OUT10:norm.weight,OUT10:proj_in.bias,OUT10:proj_in.weight,OUT10:proj_out.bias,OUT10:proj_out.weight,OUT10:transformer_blocks.0.attn1.to_k.weight,OUT10:transformer_blocks.0.attn1.to_out.0.bias,OUT10:transformer_blocks.0.attn1.to_out.0.weight,OUT10:transformer_blocks.0.attn1.to_q.weight,OUT10:transformer_blocks.0.attn1.to_v.weight,OUT10:transformer_blocks.0.attn2.to_k.weight,OUT10:transformer_blocks.0.attn2.to_out.0.bias,OUT10:transformer_blocks.0.attn2.to_out.0.weight,OUT10:transformer_blocks.0.attn2.to_q.weight,OUT10:transformer_blocks.0.attn2.to_v.weight,OUT10:transformer_blocks.0.ff.net.0.proj.bias,OUT10:transformer_blocks.0.ff.net.0.proj.weight,OUT10:transformer_blocks.0.ff.net.2.bias,OUT10:transformer_blocks.0.ff.net.2.weight,OUT10:transformer_blocks.0.norm1.bias,OUT10:transformer_blocks.0.norm1.weight,OUT10:transformer_blocks.0.norm2.bias,OUT10:transformer_blocks.0.norm2.weight,OUT10:transformer_blocks.0.norm3.bias,OUT10:transformer_blocks.0.norm3.weight
+
+OUT11:emb_layers.1.bias,OUT11:emb_layers.1.weight,OUT11:in_layers.0.bias,OUT11:in_layers.0.weight,OUT11:in_layers.2.bias,OUT11:in_layers.2.weight,OUT11:out_layers.0.bias,OUT11:out_layers.0.weight,OUT11:out_layers.3.bias,OUT11:out_layers.3.weight,OUT11:skip_connection.bias,OUT11:skip_connection.weight,OUT11:norm.bias,OUT11:norm.weight,OUT11:proj_in.bias,OUT11:proj_in.weight,OUT11:proj_out.bias,OUT11:proj_out.weight,OUT11:transformer_blocks.0.attn1.to_k.weight,OUT11:transformer_blocks.0.attn1.to_out.0.bias,OUT11:transformer_blocks.0.attn1.to_out.0.weight,OUT11:transformer_blocks.0.attn1.to_q.weight,OUT11:transformer_blocks.0.attn1.to_v.weight,OUT11:transformer_blocks.0.attn2.to_k.weight,OUT11:transformer_blocks.0.attn2.to_out.0.bias,OUT11:transformer_blocks.0.attn2.to_out.0.weight,OUT11:transformer_blocks.0.attn2.to_q.weight,OUT11:transformer_blocks.0.attn2.to_v.weight,OUT11:transformer_blocks.0.ff.net.0.proj.bias,OUT11:transformer_blocks.0.ff.net.0.proj.weight,OUT11:transformer_blocks.0.ff.net.2.bias,OUT11:transformer_blocks.0.ff.net.2.weight,OUT11:transformer_blocks.0.norm1.bias,OUT11:transformer_blocks.0.norm1.weight,OUT11:transformer_blocks.0.norm2.bias,OUT11:transformer_blocks.0.norm2.weight,OUT11:transformer_blocks.0.norm3.bias,OUT11:transformer_blocks.0.norm3.weight,OUT11:0.bias,OUT11:0.weight,OUT11:2.bias,OUT11:2.weight

microsoftexcel-supermerger/scripts/mbwpresets.txt

@@ -0,0 +1,39 @@
+preset_name	preset_weights
+GRAD_V	0,1,0.9166666667,0.8333333333,0.75,0.6666666667,0.5833333333,0.5,0.4166666667,0.3333333333,0.25,0.1666666667,0.0833333333,0,0.0833333333,0.1666666667,0.25,0.3333333333,0.4166666667,0.5,0.5833333333,0.6666666667,0.75,0.8333333333,0.9166666667,1.0
+GRAD_A	0,0,0.0833333333,0.1666666667,0.25,0.3333333333,0.4166666667,0.5,0.5833333333,0.6666666667,0.75,0.8333333333,0.9166666667,1.0,0.9166666667,0.8333333333,0.75,0.6666666667,0.5833333333,0.5,0.4166666667,0.3333333333,0.25,0.1666666667,0.0833333333,0
+FLAT_25	0,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25
+FLAT_75	0,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75
+WRAP08	0,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1
+WRAP12	0,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1
+WRAP14	0,1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1
+WRAP16	0,1,1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1
+MID12_50	0,0,0,0,0,0,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0,0,0,0,0,0
+OUT07	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1
+OUT12	0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1
+OUT12_5	0,0,0,0,0,0,0,0,0,0,0,0,0,0.5,1,1,1,1,1,1,1,1,1,1,1,1
+RING08_SOFT	0,0,0,0,0,0,0.5,1,1,1,0.5,0,0,0,0,0,0.5,1,1,1,0.5,0,0,0,0,0
+RING08_5	0,0,0,0,0,0,0,1,1,1,1,0,0,0,0,0,1,1,1,1,0,0,0,0,0,0
+RING10_5	0,0,0,0,0,0,1,1,1,1,1,0,0,0,0,0,1,1,1,1,1,0,0,0,0,0
+RING10_3	0,0,0,0,0,0,0,1,1,1,1,1,0,0,0,1,1,1,1,1,0,0,0,0,0,0
+SMOOTHSTEP	0,0,0.00506365740740741,0.0196759259259259,0.04296875,0.0740740740740741,0.112123842592593,0.15625,0.205584490740741,0.259259259259259,0.31640625,0.376157407407407,0.437644675925926,0.5,0.562355324074074,0.623842592592592,0.68359375,0.740740740740741,0.794415509259259,0.84375,0.887876157407408,0.925925925925926,0.95703125,0.980324074074074,0.994936342592593,1
+REVERSE-SMOOTHSTEP	0,1,0.994936342592593,0.980324074074074,0.95703125,0.925925925925926,0.887876157407407,0.84375,0.794415509259259,0.740740740740741,0.68359375,0.623842592592593,0.562355324074074,0.5,0.437644675925926,0.376157407407408,0.31640625,0.259259259259259,0.205584490740741,0.15625,0.112123842592592,0.0740740740740742,0.0429687499999996,0.0196759259259258,0.00506365740740744,0
+SMOOTHSTEP*2	0,0,0.0101273148148148,0.0393518518518519,0.0859375,0.148148148148148,0.224247685185185,0.3125,0.411168981481482,0.518518518518519,0.6328125,0.752314814814815,0.875289351851852,1.,0.875289351851852,0.752314814814815,0.6328125,0.518518518518519,0.411168981481481,0.3125,0.224247685185184,0.148148148148148,0.0859375,0.0393518518518512,0.0101273148148153,0
+R_SMOOTHSTEP*2	0,1,0.989872685185185,0.960648148148148,0.9140625,0.851851851851852,0.775752314814815,0.6875,0.588831018518519,0.481481481481481,0.3671875,0.247685185185185,0.124710648148148,0.,0.124710648148148,0.247685185185185,0.3671875,0.481481481481481,0.588831018518519,0.6875,0.775752314814816,0.851851851851852,0.9140625,0.960648148148149,0.989872685185185,1
+SMOOTHSTEP*3	0,0,0.0151909722222222,0.0590277777777778,0.12890625,0.222222222222222,0.336371527777778,0.46875,0.616753472222222,0.777777777777778,0.94921875,0.871527777777778,0.687065972222222,0.5,0.312934027777778,0.128472222222222,0.0507812500000004,0.222222222222222,0.383246527777778,0.53125,0.663628472222223,0.777777777777778,0.87109375,0.940972222222222,0.984809027777777,1
+R_SMOOTHSTEP*3	0,1,0.984809027777778,0.940972222222222,0.87109375,0.777777777777778,0.663628472222222,0.53125,0.383246527777778,0.222222222222222,0.05078125,0.128472222222222,0.312934027777778,0.5,0.687065972222222,0.871527777777778,0.94921875,0.777777777777778,0.616753472222222,0.46875,0.336371527777777,0.222222222222222,0.12890625,0.0590277777777777,0.0151909722222232,0
+SMOOTHSTEP*4	0,0,0.0202546296296296,0.0787037037037037,0.171875,0.296296296296296,0.44849537037037,0.625,0.822337962962963,0.962962962962963,0.734375,0.49537037037037,0.249421296296296,0.,0.249421296296296,0.495370370370371,0.734375000000001,0.962962962962963,0.822337962962962,0.625,0.448495370370369,0.296296296296297,0.171875,0.0787037037037024,0.0202546296296307,0
+R_SMOOTHSTEP*4	0,1,0.97974537037037,0.921296296296296,0.828125,0.703703703703704,0.55150462962963,0.375,0.177662037037037,0.0370370370370372,0.265625,0.50462962962963,0.750578703703704,1.,0.750578703703704,0.504629629629629,0.265624999999999,0.0370370370370372,0.177662037037038,0.375,0.551504629629631,0.703703703703703,0.828125,0.921296296296298,0.979745370370369,1
+SMOOTHSTEP/2	0,0,0.0196759259259259,0.0740740740740741,0.15625,0.259259259259259,0.376157407407407,0.5,0.623842592592593,0.740740740740741,0.84375,0.925925925925926,0.980324074074074,1.,0.980324074074074,0.925925925925926,0.84375,0.740740740740741,0.623842592592593,0.5,0.376157407407407,0.259259259259259,0.15625,0.0740740740740741,0.0196759259259259,0
+R_SMOOTHSTEP/2	0,1,0.980324074074074,0.925925925925926,0.84375,0.740740740740741,0.623842592592593,0.5,0.376157407407407,0.259259259259259,0.15625,0.0740740740740742,0.0196759259259256,0.,0.0196759259259256,0.0740740740740742,0.15625,0.259259259259259,0.376157407407407,0.5,0.623842592592593,0.740740740740741,0.84375,0.925925925925926,0.980324074074074,1
+SMOOTHSTEP/3	0,0,0.04296875,0.15625,0.31640625,0.5,0.68359375,0.84375,0.95703125,1.,0.95703125,0.84375,0.68359375,0.5,0.31640625,0.15625,0.04296875,0.,0.04296875,0.15625,0.31640625,0.5,0.68359375,0.84375,0.95703125,1
+R_SMOOTHSTEP/3	0,1,0.95703125,0.84375,0.68359375,0.5,0.31640625,0.15625,0.04296875,0.,0.04296875,0.15625,0.31640625,0.5,0.68359375,0.84375,0.95703125,1.,0.95703125,0.84375,0.68359375,0.5,0.31640625,0.15625,0.04296875,0
+SMOOTHSTEP/4	0,0,0.0740740740740741,0.259259259259259,0.5,0.740740740740741,0.925925925925926,1.,0.925925925925926,0.740740740740741,0.5,0.259259259259259,0.0740740740740741,0.,0.0740740740740741,0.259259259259259,0.5,0.740740740740741,0.925925925925926,1.,0.925925925925926,0.740740740740741,0.5,0.259259259259259,0.0740740740740741,0
+R_SMOOTHSTEP/4	0,1,0.925925925925926,0.740740740740741,0.5,0.259259259259259,0.0740740740740742,0.,0.0740740740740742,0.259259259259259,0.5,0.740740740740741,0.925925925925926,1.,0.925925925925926,0.740740740740741,0.5,0.259259259259259,0.0740740740740742,0.,0.0740740740740742,0.259259259259259,0.5,0.740740740740741,0.925925925925926,1
+COSINE	0,1,0.995722430686905,0.982962913144534,0.961939766255643,0.933012701892219,0.896676670145617,0.853553390593274,0.80438071450436,0.75,0.691341716182545,0.62940952255126,0.565263096110026,0.5,0.434736903889974,0.37059047744874,0.308658283817455,0.25,0.195619285495639,0.146446609406726,0.103323329854382,0.0669872981077805,0.0380602337443566,0.0170370868554658,0.00427756931309475,0
+REVERSE_COSINE	0,0,0.00427756931309475,0.0170370868554659,0.0380602337443566,0.0669872981077808,0.103323329854383,0.146446609406726,0.19561928549564,0.25,0.308658283817455,0.37059047744874,0.434736903889974,0.5,0.565263096110026,0.62940952255126,0.691341716182545,0.75,0.804380714504361,0.853553390593274,0.896676670145618,0.933012701892219,0.961939766255643,0.982962913144534,0.995722430686905,1
+TRUE_CUBIC_HERMITE	0,0,0.199031876929012,0.325761959876543,0.424641927083333,0.498456790123457,0.549991560570988,0.58203125,0.597360869984568,0.598765432098765,0.589029947916667,0.570939429012346,0.547278886959876,0.520833333333333,0.49438777970679,0.470727237654321,0.45263671875,0.442901234567901,0.444305796682099,0.459635416666667,0.491675106095678,0.543209876543211,0.617024739583333,0.715904706790124,0.842634789737655,1
+TRUE_REVERSE_CUBIC_HERMITE	0,1,0.800968123070988,0.674238040123457,0.575358072916667,0.501543209876543,0.450008439429012,0.41796875,0.402639130015432,0.401234567901235,0.410970052083333,0.429060570987654,0.452721113040124,0.479166666666667,0.50561222029321,0.529272762345679,0.54736328125,0.557098765432099,0.555694203317901,0.540364583333333,0.508324893904322,0.456790123456789,0.382975260416667,0.284095293209876,0.157365210262345,0
+FAKE_CUBIC_HERMITE	0,0,0.157576195987654,0.28491512345679,0.384765625,0.459876543209877,0.512996720679012,0.546875,0.564260223765432,0.567901234567901,0.560546875,0.544945987654321,0.523847415123457,0.5,0.476152584876543,0.455054012345679,0.439453125,0.432098765432099,0.435739776234568,0.453125,0.487003279320987,0.540123456790124,0.615234375,0.71508487654321,0.842423804012347,1
+FAKE_REVERSE_CUBIC_HERMITE	0,1,0.842423804012346,0.71508487654321,0.615234375,0.540123456790123,0.487003279320988,0.453125,0.435739776234568,0.432098765432099,0.439453125,0.455054012345679,0.476152584876543,0.5,0.523847415123457,0.544945987654321,0.560546875,0.567901234567901,0.564260223765432,0.546875,0.512996720679013,0.459876543209876,0.384765625,0.28491512345679,0.157576195987653,0
+ALL_A	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
+ALL_B	1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1

microsoftexcel-supermerger/scripts/mbwpresets_master.txt

@@ -0,0 +1,39 @@
+preset_name	preset_weights
+GRAD_V	0,1,0.9166666667,0.8333333333,0.75,0.6666666667,0.5833333333,0.5,0.4166666667,0.3333333333,0.25,0.1666666667,0.0833333333,0,0.0833333333,0.1666666667,0.25,0.3333333333,0.4166666667,0.5,0.5833333333,0.6666666667,0.75,0.8333333333,0.9166666667,1.0
+GRAD_A	0,0,0.0833333333,0.1666666667,0.25,0.3333333333,0.4166666667,0.5,0.5833333333,0.6666666667,0.75,0.8333333333,0.9166666667,1.0,0.9166666667,0.8333333333,0.75,0.6666666667,0.5833333333,0.5,0.4166666667,0.3333333333,0.25,0.1666666667,0.0833333333,0
+FLAT_25	0,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25
+FLAT_75	0,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75,0.75
+WRAP08	0,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1
+WRAP12	0,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1
+WRAP14	0,1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1
+WRAP16	0,1,1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1
+MID12_50	0,0,0,0,0,0,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0,0,0,0,0,0
+OUT07	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1
+OUT12	0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1
+OUT12_5	0,0,0,0,0,0,0,0,0,0,0,0,0,0.5,1,1,1,1,1,1,1,1,1,1,1,1
+RING08_SOFT	0,0,0,0,0,0,0.5,1,1,1,0.5,0,0,0,0,0,0.5,1,1,1,0.5,0,0,0,0,0
+RING08_5	0,0,0,0,0,0,0,1,1,1,1,0,0,0,0,0,1,1,1,1,0,0,0,0,0,0
+RING10_5	0,0,0,0,0,0,1,1,1,1,1,0,0,0,0,0,1,1,1,1,1,0,0,0,0,0
+RING10_3	0,0,0,0,0,0,0,1,1,1,1,1,0,0,0,1,1,1,1,1,0,0,0,0,0,0
+SMOOTHSTEP	0,0,0.00506365740740741,0.0196759259259259,0.04296875,0.0740740740740741,0.112123842592593,0.15625,0.205584490740741,0.259259259259259,0.31640625,0.376157407407407,0.437644675925926,0.5,0.562355324074074,0.623842592592592,0.68359375,0.740740740740741,0.794415509259259,0.84375,0.887876157407408,0.925925925925926,0.95703125,0.980324074074074,0.994936342592593,1
+REVERSE-SMOOTHSTEP	0,1,0.994936342592593,0.980324074074074,0.95703125,0.925925925925926,0.887876157407407,0.84375,0.794415509259259,0.740740740740741,0.68359375,0.623842592592593,0.562355324074074,0.5,0.437644675925926,0.376157407407408,0.31640625,0.259259259259259,0.205584490740741,0.15625,0.112123842592592,0.0740740740740742,0.0429687499999996,0.0196759259259258,0.00506365740740744,0
+SMOOTHSTEP*2	0,0,0.0101273148148148,0.0393518518518519,0.0859375,0.148148148148148,0.224247685185185,0.3125,0.411168981481482,0.518518518518519,0.6328125,0.752314814814815,0.875289351851852,1.,0.875289351851852,0.752314814814815,0.6328125,0.518518518518519,0.411168981481481,0.3125,0.224247685185184,0.148148148148148,0.0859375,0.0393518518518512,0.0101273148148153,0
+R_SMOOTHSTEP*2	0,1,0.989872685185185,0.960648148148148,0.9140625,0.851851851851852,0.775752314814815,0.6875,0.588831018518519,0.481481481481481,0.3671875,0.247685185185185,0.124710648148148,0.,0.124710648148148,0.247685185185185,0.3671875,0.481481481481481,0.588831018518519,0.6875,0.775752314814816,0.851851851851852,0.9140625,0.960648148148149,0.989872685185185,1
+SMOOTHSTEP*3	0,0,0.0151909722222222,0.0590277777777778,0.12890625,0.222222222222222,0.336371527777778,0.46875,0.616753472222222,0.777777777777778,0.94921875,0.871527777777778,0.687065972222222,0.5,0.312934027777778,0.128472222222222,0.0507812500000004,0.222222222222222,0.383246527777778,0.53125,0.663628472222223,0.777777777777778,0.87109375,0.940972222222222,0.984809027777777,1
+R_SMOOTHSTEP*3	0,1,0.984809027777778,0.940972222222222,0.87109375,0.777777777777778,0.663628472222222,0.53125,0.383246527777778,0.222222222222222,0.05078125,0.128472222222222,0.312934027777778,0.5,0.687065972222222,0.871527777777778,0.94921875,0.777777777777778,0.616753472222222,0.46875,0.336371527777777,0.222222222222222,0.12890625,0.0590277777777777,0.0151909722222232,0
+SMOOTHSTEP*4	0,0,0.0202546296296296,0.0787037037037037,0.171875,0.296296296296296,0.44849537037037,0.625,0.822337962962963,0.962962962962963,0.734375,0.49537037037037,0.249421296296296,0.,0.249421296296296,0.495370370370371,0.734375000000001,0.962962962962963,0.822337962962962,0.625,0.448495370370369,0.296296296296297,0.171875,0.0787037037037024,0.0202546296296307,0
+R_SMOOTHSTEP*4	0,1,0.97974537037037,0.921296296296296,0.828125,0.703703703703704,0.55150462962963,0.375,0.177662037037037,0.0370370370370372,0.265625,0.50462962962963,0.750578703703704,1.,0.750578703703704,0.504629629629629,0.265624999999999,0.0370370370370372,0.177662037037038,0.375,0.551504629629631,0.703703703703703,0.828125,0.921296296296298,0.979745370370369,1
+SMOOTHSTEP/2	0,0,0.0196759259259259,0.0740740740740741,0.15625,0.259259259259259,0.376157407407407,0.5,0.623842592592593,0.740740740740741,0.84375,0.925925925925926,0.980324074074074,1.,0.980324074074074,0.925925925925926,0.84375,0.740740740740741,0.623842592592593,0.5,0.376157407407407,0.259259259259259,0.15625,0.0740740740740741,0.0196759259259259,0
+R_SMOOTHSTEP/2	0,1,0.980324074074074,0.925925925925926,0.84375,0.740740740740741,0.623842592592593,0.5,0.376157407407407,0.259259259259259,0.15625,0.0740740740740742,0.0196759259259256,0.,0.0196759259259256,0.0740740740740742,0.15625,0.259259259259259,0.376157407407407,0.5,0.623842592592593,0.740740740740741,0.84375,0.925925925925926,0.980324074074074,1
+SMOOTHSTEP/3	0,0,0.04296875,0.15625,0.31640625,0.5,0.68359375,0.84375,0.95703125,1.,0.95703125,0.84375,0.68359375,0.5,0.31640625,0.15625,0.04296875,0.,0.04296875,0.15625,0.31640625,0.5,0.68359375,0.84375,0.95703125,1
+R_SMOOTHSTEP/3	0,1,0.95703125,0.84375,0.68359375,0.5,0.31640625,0.15625,0.04296875,0.,0.04296875,0.15625,0.31640625,0.5,0.68359375,0.84375,0.95703125,1.,0.95703125,0.84375,0.68359375,0.5,0.31640625,0.15625,0.04296875,0
+SMOOTHSTEP/4	0,0,0.0740740740740741,0.259259259259259,0.5,0.740740740740741,0.925925925925926,1.,0.925925925925926,0.740740740740741,0.5,0.259259259259259,0.0740740740740741,0.,0.0740740740740741,0.259259259259259,0.5,0.740740740740741,0.925925925925926,1.,0.925925925925926,0.740740740740741,0.5,0.259259259259259,0.0740740740740741,0
+R_SMOOTHSTEP/4	0,1,0.925925925925926,0.740740740740741,0.5,0.259259259259259,0.0740740740740742,0.,0.0740740740740742,0.259259259259259,0.5,0.740740740740741,0.925925925925926,1.,0.925925925925926,0.740740740740741,0.5,0.259259259259259,0.0740740740740742,0.,0.0740740740740742,0.259259259259259,0.5,0.740740740740741,0.925925925925926,1
+COSINE	0,1,0.995722430686905,0.982962913144534,0.961939766255643,0.933012701892219,0.896676670145617,0.853553390593274,0.80438071450436,0.75,0.691341716182545,0.62940952255126,0.565263096110026,0.5,0.434736903889974,0.37059047744874,0.308658283817455,0.25,0.195619285495639,0.146446609406726,0.103323329854382,0.0669872981077805,0.0380602337443566,0.0170370868554658,0.00427756931309475,0
+REVERSE_COSINE	0,0,0.00427756931309475,0.0170370868554659,0.0380602337443566,0.0669872981077808,0.103323329854383,0.146446609406726,0.19561928549564,0.25,0.308658283817455,0.37059047744874,0.434736903889974,0.5,0.565263096110026,0.62940952255126,0.691341716182545,0.75,0.804380714504361,0.853553390593274,0.896676670145618,0.933012701892219,0.961939766255643,0.982962913144534,0.995722430686905,1
+TRUE_CUBIC_HERMITE	0,0,0.199031876929012,0.325761959876543,0.424641927083333,0.498456790123457,0.549991560570988,0.58203125,0.597360869984568,0.598765432098765,0.589029947916667,0.570939429012346,0.547278886959876,0.520833333333333,0.49438777970679,0.470727237654321,0.45263671875,0.442901234567901,0.444305796682099,0.459635416666667,0.491675106095678,0.543209876543211,0.617024739583333,0.715904706790124,0.842634789737655,1
+TRUE_REVERSE_CUBIC_HERMITE	0,1,0.800968123070988,0.674238040123457,0.575358072916667,0.501543209876543,0.450008439429012,0.41796875,0.402639130015432,0.401234567901235,0.410970052083333,0.429060570987654,0.452721113040124,0.479166666666667,0.50561222029321,0.529272762345679,0.54736328125,0.557098765432099,0.555694203317901,0.540364583333333,0.508324893904322,0.456790123456789,0.382975260416667,0.284095293209876,0.157365210262345,0
+FAKE_CUBIC_HERMITE	0,0,0.157576195987654,0.28491512345679,0.384765625,0.459876543209877,0.512996720679012,0.546875,0.564260223765432,0.567901234567901,0.560546875,0.544945987654321,0.523847415123457,0.5,0.476152584876543,0.455054012345679,0.439453125,0.432098765432099,0.435739776234568,0.453125,0.487003279320987,0.540123456790124,0.615234375,0.71508487654321,0.842423804012347,1
+FAKE_REVERSE_CUBIC_HERMITE	0,1,0.842423804012346,0.71508487654321,0.615234375,0.540123456790123,0.487003279320988,0.453125,0.435739776234568,0.432098765432099,0.439453125,0.455054012345679,0.476152584876543,0.5,0.523847415123457,0.544945987654321,0.560546875,0.567901234567901,0.564260223765432,0.546875,0.512996720679013,0.459876543209876,0.384765625,0.28491512345679,0.157576195987653,0
+ALL_A	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
+ALL_B	1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1

microsoftexcel-supermerger/scripts/mergers/mergers.py

@@ -0,0 +1,699 @@
+from linecache import clearcache
+
+import os
+import gc
+import numpy as np
+import os.path
+import re
+import torch
+import tqdm
+import datetime
+import csv
+import json
+import torch.nn as nn
+import scipy.ndimage
+from scipy.ndimage.filters import median_filter as filter
+from PIL import Image, ImageFont, ImageDraw
+from tqdm import tqdm
+from modules import shared, processing, sd_models, images, sd_samplers,scripts
+from modules.ui import  plaintext_to_html
+from modules.shared import opts
+from modules.processing import create_infotext,Processed
+from modules.sd_models import  load_model,checkpoints_loaded
+from scripts.mergers.model_util import usemodelgen,filenamecutter,savemodel
+
+from inspect import currentframe
+
+stopmerge = False
+
+def freezemtime():
+    global stopmerge
+    stopmerge = True
+
+mergedmodel=[]
+TYPESEG = ["none","alpha","beta (if Triple or Twice is not selected,Twice automatically enable)","alpha and beta","seed", "mbw alpha","mbw beta","mbw alpha and beta", "model_A","model_B","model_C","pinpoint blocks (alpha or beta must be selected for another axis)","elemental","pinpoint element","effective elemental checker","tensors","calcmode","prompt"]
+TYPES = ["none","alpha","beta","alpha and beta","seed", "mbw alpha ","mbw beta","mbw alpha and beta", "model_A","model_B","model_C","pinpoint blocks","elemental","pinpoint element","effective","tensor","calcmode","prompt"]
+MODES=["Weight" ,"Add" ,"Triple","Twice"]
+SAVEMODES=["save model", "overwrite"]
+#type[0:aplha,1:beta,2:seed,3:mbw,4:model_A,5:model_B,6:model_C]
+#msettings=[0 weights_a,1 weights_b,2 model_a,3 model_b,4 model_c,5 base_alpha,6 base_beta,7 mode,8 useblocks,9 custom_name,10 save_sets,11 id_sets,12 wpresets]
+#id sets "image", "PNG info","XY grid"
+
+hear = False
+hearm = False
+non4 = [None]*4
+
+def caster(news,hear):
+    if hear: print(news)
+
+def casterr(*args,hear=hear):
+    if hear:
+        names = {id(v): k for k, v in currentframe().f_back.f_locals.items()}
+        print('\n'.join([names.get(id(arg), '???') + ' = ' + repr(arg) for arg in args]))
+    
+  #msettings=[weights_a,weights_b,model_a,model_b,model_c,device,base_alpha,base_beta,mode,loranames,useblocks,custom_name,save_sets,id_sets,wpresets,deep]  
+def smergegen(weights_a,weights_b,model_a,model_b,model_c,base_alpha,base_beta,mode,
+                       calcmode,useblocks,custom_name,save_sets,id_sets,wpresets,deep,tensor,
+                       esettings,
+                       prompt,nprompt,steps,sampler,cfg,seed,w,h,
+                       hireson,hrupscaler,hr2ndsteps,denoise_str,hr_scale,batch_size,
+                       currentmodel,imggen):
+
+    deepprint  = True if "print change" in esettings else False
+
+    result,currentmodel,modelid,theta_0,metadata = smerge(
+                        weights_a,weights_b,model_a,model_b,model_c,base_alpha,base_beta,mode,calcmode,
+                        useblocks,custom_name,save_sets,id_sets,wpresets,deep,tensor,deepprint=deepprint
+                        )
+
+    if "ERROR" in result or "STOPPED" in result: 
+        return result,"not loaded",*non4
+
+    usemodelgen(theta_0,model_a,currentmodel)
+
+    save = True if SAVEMODES[0] in save_sets else False
+
+    result = savemodel(theta_0,currentmodel,custom_name,save_sets,model_a,metadata) if save else "Merged model loaded:"+currentmodel
+    del theta_0
+    gc.collect()
+
+    if imggen :
+        images = simggen(prompt,nprompt,steps,sampler,cfg,seed,w,h,hireson,hrupscaler,hr2ndsteps,denoise_str,hr_scale,batch_size,currentmodel,id_sets,modelid)
+        return result,currentmodel,*images[:4]
+    else:
+        return result,currentmodel
+
+NUM_INPUT_BLOCKS = 12
+NUM_MID_BLOCK = 1
+NUM_OUTPUT_BLOCKS = 12
+NUM_TOTAL_BLOCKS = NUM_INPUT_BLOCKS + NUM_MID_BLOCK + NUM_OUTPUT_BLOCKS
+blockid=["BASE","IN00","IN01","IN02","IN03","IN04","IN05","IN06","IN07","IN08","IN09","IN10","IN11","M00","OUT00","OUT01","OUT02","OUT03","OUT04","OUT05","OUT06","OUT07","OUT08","OUT09","OUT10","OUT11"]
+     
+def smerge(weights_a,weights_b,model_a,model_b,model_c,base_alpha,base_beta,mode,calcmode,
+                useblocks,custom_name,save_sets,id_sets,wpresets,deep,tensor,deepprint = False):
+    caster("merge start",hearm)
+    global hear,mergedmodel,stopmerge
+    stopmerge = False
+
+    gc.collect()
+
+    # for from file
+    if type(useblocks) is str:
+        useblocks = True if useblocks =="True" else False
+    if type(base_alpha) == str:base_alpha = float(base_alpha)
+    if type(base_beta) == str:base_beta  = float(base_beta)
+
+    weights_a_orig = weights_a
+    weights_b_orig = weights_b
+
+    # preset to weights
+    if wpresets != False and useblocks:
+        weights_a = wpreseter(weights_a,wpresets)
+        weights_b = wpreseter(weights_b,wpresets)
+
+    # mode select booleans
+    save = True if SAVEMODES[0] in save_sets else False
+    usebeta = MODES[2] in mode or MODES[3] in mode or calcmode == "tensor"
+    save_metadata = "save metadata" in save_sets
+    metadata = {"format": "pt"}
+
+    if not useblocks:
+        weights_a = weights_b = ""
+    #for save log and save current model
+    mergedmodel =[weights_a,weights_b,
+                            hashfromname(model_a),hashfromname(model_b),hashfromname(model_c),
+                            base_alpha,base_beta,mode,useblocks,custom_name,save_sets,id_sets,deep,calcmode,tensor].copy()
+    
+    model_a = namefromhash(model_a)
+    model_b = namefromhash(model_b)
+    model_c = namefromhash(model_c)
+    theta_2 = {}
+
+    caster(mergedmodel,False)
+
+    if len(deep) > 0:
+        deep = deep.replace("\n",",")
+        deep = deep.split(",")
+
+    #format check
+    if model_a =="" or model_b =="" or ((not MODES[0] in mode) and model_c=="") : 
+        return "ERROR: Necessary model is not selected",*non4
+    
+    #for MBW text to list
+    if useblocks:
+        weights_a_t=weights_a.split(',',1)
+        weights_b_t=weights_b.split(',',1)
+        base_alpha  = float(weights_a_t[0])    
+        weights_a = [float(w) for w in weights_a_t[1].split(',')]
+        caster(f"from {weights_a_t}, alpha = {base_alpha},weights_a ={weights_a}",hearm)
+        if len(weights_a) != 25:return f"ERROR: weights alpha value must be {26}.",*non4
+        if usebeta:
+            base_beta = float(weights_b_t[0]) 
+            weights_b = [float(w) for w in weights_b_t[1].split(',')]
+            caster(f"from {weights_b_t}, beta = {base_beta},weights_a ={weights_b}",hearm)
+            if len(weights_b) != 25: return f"ERROR: weights beta value must be {26}.",*non4
+        
+    caster("model load start",hearm)
+
+    print(f"  model A  \t: {model_a}")
+    print(f"  model B  \t: {model_b}")
+    print(f"  model C  \t: {model_c}")
+    print(f"  alpha,beta\t: {base_alpha,base_beta}")
+    print(f"  weights_alpha\t: {weights_a}")
+    print(f"  weights_beta\t: {weights_b}")
+    print(f"  mode\t\t: {mode}")
+    print(f"  MBW \t\t: {useblocks}")
+    print(f"  CalcMode \t: {calcmode}")
+    print(f"  Elemental \t: {deep}")
+    print(f"  Tensors \t: {tensor}")
+
+    theta_1=load_model_weights_m(model_b,False,True,save).copy()
+
+    if MODES[1] in mode:#Add
+        if stopmerge: return "STOPPED", *non4
+        theta_2 = load_model_weights_m(model_c,False,False,save).copy()
+        for key in tqdm(theta_1.keys()):
+            if 'model' in key:
+                if key in theta_2:
+                    t2 = theta_2.get(key, torch.zeros_like(theta_1[key]))
+                    theta_1[key] = theta_1[key]- t2
+                else:
+                    theta_1[key] = torch.zeros_like(theta_1[key])
+        del theta_2
+
+    if stopmerge: return "STOPPED", *non4
+    
+    if calcmode == "tensor":
+        theta_t = load_model_weights_m(model_a,True,False,save).copy()
+        theta_0 ={}
+        for key in theta_t:
+            theta_0[key] = theta_t[key].clone()
+        del theta_t
+    else:
+        theta_0=load_model_weights_m(model_a,True,False,save).copy()
+
+
+    if MODES[2] in mode or MODES[3] in mode:#Tripe or Twice
+        theta_2 = load_model_weights_m(model_c,False,False,save).copy()
+
+    alpha = base_alpha
+    beta = base_beta
+
+    re_inp = re.compile(r'\.input_blocks\.(\d+)\.')  # 12
+    re_mid = re.compile(r'\.middle_block\.(\d+)\.')  # 1
+    re_out = re.compile(r'\.output_blocks\.(\d+)\.') # 12
+
+    chckpoint_dict_skip_on_merge = ["cond_stage_model.transformer.text_model.embeddings.position_ids"]
+    count_target_of_basealpha = 0
+
+    if calcmode =="cosineA": #favors modelA's structure with details from B
+        if stopmerge: return "STOPPED", *non4
+        sim = torch.nn.CosineSimilarity(dim=0)
+        sims = np.array([], dtype=np.float64)
+        for key in (tqdm(theta_0.keys(), desc="Stage 0/2")):
+            # skip VAE model parameters to get better results
+            if "first_stage_model" in key: continue
+            if "model" in key and key in theta_1:
+                theta_0_norm = nn.functional.normalize(theta_0[key].to(torch.float32), p=2, dim=0)
+                theta_1_norm = nn.functional.normalize(theta_1[key].to(torch.float32), p=2, dim=0)
+                simab = sim(theta_0_norm, theta_1_norm)
+                sims = np.append(sims,simab.numpy())
+        sims = sims[~np.isnan(sims)]
+        sims = np.delete(sims, np.where(sims<np.percentile(sims, 1 ,method = 'midpoint')))
+        sims = np.delete(sims, np.where(sims>np.percentile(sims, 99 ,method = 'midpoint')))
+
+    if calcmode =="cosineB": #favors modelB's structure with details from A
+        if stopmerge: return "STOPPED", *non4
+        sim = torch.nn.CosineSimilarity(dim=0)
+        sims = np.array([], dtype=np.float64)
+        for key in (tqdm(theta_0.keys(), desc="Stage 0/2")):
+            # skip VAE model parameters to get better results
+            if "first_stage_model" in key: continue
+            if "model" in key and key in theta_1:
+                simab = sim(theta_0[key].to(torch.float32), theta_1[key].to(torch.float32))
+                dot_product = torch.dot(theta_0[key].view(-1).to(torch.float32), theta_1[key].view(-1).to(torch.float32))
+                magnitude_similarity = dot_product / (torch.norm(theta_0[key].to(torch.float32)) * torch.norm(theta_1[key].to(torch.float32)))
+                combined_similarity = (simab + magnitude_similarity) / 2.0
+                sims = np.append(sims, combined_similarity.numpy())
+        sims = sims[~np.isnan(sims)]
+        sims = np.delete(sims, np.where(sims < np.percentile(sims, 1, method='midpoint')))
+        sims = np.delete(sims, np.where(sims > np.percentile(sims, 99, method='midpoint')))
+
+    for key in (tqdm(theta_0.keys(), desc="Stage 1/2") if not False else theta_0.keys()):
+        if stopmerge: return "STOPPED", *non4
+        if "model" in key and key in theta_1:
+            if usebeta and (not key in theta_2) and (not theta_2 == {}) :
+                continue
+
+            weight_index = -1
+            current_alpha = alpha
+            current_beta = beta
+
+            if key in chckpoint_dict_skip_on_merge:
+                continue
+
+            # check weighted and U-Net or not
+            if weights_a is not None and 'model.diffusion_model.' in key:
+                # check block index
+                weight_index = -1
+
+                if 'time_embed' in key:
+                    weight_index = 0                # before input blocks
+                elif '.out.' in key:
+                    weight_index = NUM_TOTAL_BLOCKS - 1     # after output blocks
+                else:
+                    m = re_inp.search(key)
+                    if m:
+                        inp_idx = int(m.groups()[0])
+                        weight_index = inp_idx
+                    else:
+                        m = re_mid.search(key)
+                        if m:
+                            weight_index = NUM_INPUT_BLOCKS
+                        else:
+                            m = re_out.search(key)
+                            if m:
+                                out_idx = int(m.groups()[0])
+                                weight_index = NUM_INPUT_BLOCKS + NUM_MID_BLOCK + out_idx
+
+                if weight_index >= NUM_TOTAL_BLOCKS:
+                    print(f"ERROR: illegal block index: {key}")
+                    return f"ERROR: illegal block index: {key}",*non4
+                
+                if weight_index >= 0 and useblocks:
+                    current_alpha = weights_a[weight_index]
+                    if usebeta: current_beta = weights_b[weight_index]
+            else:
+                count_target_of_basealpha = count_target_of_basealpha + 1
+
+            if len(deep) > 0:
+                skey = key + blockid[weight_index+1]
+                for d in deep:
+                    if d.count(":") != 2 :continue
+                    dbs,dws,dr = d.split(":")[0],d.split(":")[1],d.split(":")[2]
+                    dbs,dws = dbs.split(" "), dws.split(" ")
+                    dbn,dbs = (True,dbs[1:]) if dbs[0] == "NOT" else (False,dbs)
+                    dwn,dws = (True,dws[1:]) if dws[0] == "NOT" else (False,dws)
+                    flag = dbn
+                    for db in dbs:
+                        if db in skey:
+                            flag = not dbn
+                    if flag:flag = dwn
+                    else:continue
+                    for dw in dws:
+                        if dw in skey:
+                            flag = not dwn
+                    if flag:
+                        dr = float(dr)
+                        if deepprint :print(dbs,dws,key,dr)
+                        current_alpha = dr
+
+            if calcmode == "normal":
+                if MODES[1] in mode:#Add
+                    caster(f"model A[{key}] +  {current_alpha} + * (model B - model C)[{key}]",hear)
+                    theta_0[key] = theta_0[key] + current_alpha * theta_1[key]
+                elif MODES[2] in mode:#Triple
+                    caster(f"model A[{key}] +  {1-current_alpha-current_beta} +  model B[{key}]*{current_alpha} + model C[{key}]*{current_beta}",hear)
+                    theta_0[key] = (1 - current_alpha-current_beta) * theta_0[key] + current_alpha * theta_1[key]+current_beta * theta_2[key]
+                elif MODES[3] in mode:#Twice
+                    caster(f"model A[{key}] +  {1-current_alpha} + * model B[{key}]*{alpha}",hear)
+                    caster(f"model A+B[{key}] +  {1-current_beta} + * model C[{key}]*{beta}",hear)
+                    theta_0[key] = (1 - current_alpha) * theta_0[key] + current_alpha * theta_1[key]
+                    theta_0[key] = (1 - current_beta) * theta_0[key] + current_beta * theta_2[key]
+                else:#Weight
+                    if current_alpha == 1:
+                        caster(f"alpha = 0,model A[{key}=model B[{key}",hear)
+                        theta_0[key] = theta_1[key]
+                    elif current_alpha !=0:
+                        caster(f"model A[{key}] +  {1-current_alpha} + * (model B)[{key}]*{alpha}",hear)
+                        theta_0[key] = (1 - current_alpha) * theta_0[key] + current_alpha * theta_1[key]
+
+            elif calcmode == "cosineA": #favors modelA's structure with details from B
+                # skip VAE model parameters to get better results
+                if "first_stage_model" in key: continue
+                if "model" in key and key in theta_0:
+                    # Normalize the vectors before merging
+                    theta_0_norm = nn.functional.normalize(theta_0[key].to(torch.float32), p=2, dim=0)
+                    theta_1_norm = nn.functional.normalize(theta_1[key].to(torch.float32), p=2, dim=0)
+                    simab = sim(theta_0_norm, theta_1_norm)
+                    dot_product = torch.dot(theta_0_norm.view(-1), theta_1_norm.view(-1))
+                    magnitude_similarity = dot_product / (torch.norm(theta_0_norm) * torch.norm(theta_1_norm))
+                    combined_similarity = (simab + magnitude_similarity) / 2.0
+                    k = (combined_similarity - sims.min()) / (sims.max() - sims.min())
+                    k = k - current_alpha
+                    k = k.clip(min=.0,max=1.)
+                    caster(f"model A[{key}] +  {1-k} + * (model B)[{key}]*{k}",hear)
+                    theta_0[key] = theta_1[key] * (1 - k) + theta_0[key] * k
+
+            elif calcmode == "cosineB": #favors modelB's structure with details from A
+                # skip VAE model parameters to get better results
+                if "first_stage_model" in key: continue
+                if "model" in key and key in theta_0:
+                    simab = sim(theta_0[key].to(torch.float32), theta_1[key].to(torch.float32))
+                    dot_product = torch.dot(theta_0[key].view(-1).to(torch.float32), theta_1[key].view(-1).to(torch.float32))
+                    magnitude_similarity = dot_product / (torch.norm(theta_0[key].to(torch.float32)) * torch.norm(theta_1[key].to(torch.float32)))
+                    combined_similarity = (simab + magnitude_similarity) / 2.0
+                    k = (combined_similarity - sims.min()) / (sims.max() - sims.min())
+                    k = k - current_alpha
+                    k = k.clip(min=.0,max=1.)
+                    caster(f"model A[{key}] +  {1-k} + * (model B)[{key}]*{k}",hear)
+                    theta_0[key] = theta_1[key] * (1 - k) + theta_0[key] * k
+
+            elif calcmode == "smoothAdd":
+                caster(f"model A[{key}] +  {current_alpha} + * (model B - model C)[{key}]", hear)
+                # Apply median filter to the weight differences
+                filtered_diff = scipy.ndimage.median_filter(theta_1[key].to(torch.float32).cpu().numpy(), size=3)
+                # Apply Gaussian filter to the filtered differences
+                filtered_diff = scipy.ndimage.gaussian_filter(filtered_diff, sigma=1)
+                theta_1[key] = torch.tensor(filtered_diff)
+                # Add the filtered differences to the original weights
+                theta_0[key] = theta_0[key] + current_alpha * theta_1[key]
+
+            elif calcmode == "tensor":
+                dim = theta_0[key].dim()
+                if dim == 0 : continue
+                if current_alpha+current_beta <= 1 :
+                    talphas = int(theta_0[key].shape[0]*(current_beta))
+                    talphae = int(theta_0[key].shape[0]*(current_alpha+current_beta))
+                    if dim == 1:
+                        theta_0[key][talphas:talphae] = theta_1[key][talphas:talphae].clone()
+
+                    elif dim == 2:
+                        theta_0[key][talphas:talphae,:] = theta_1[key][talphas:talphae,:].clone()
+
+                    elif dim == 3:
+                        theta_0[key][talphas:talphae,:,:] = theta_1[key][talphas:talphae,:,:].clone()
+
+                    elif dim == 4:
+                        theta_0[key][talphas:talphae,:,:,:] = theta_1[key][talphas:talphae,:,:,:].clone()
+
+                else:
+                    talphas = int(theta_0[key].shape[0]*(current_alpha+current_beta-1))
+                    talphae = int(theta_0[key].shape[0]*(current_beta))
+                    theta_t = theta_1[key].clone()
+                    if dim == 1:
+                        theta_t[talphas:talphae] = theta_0[key][talphas:talphae].clone()
+
+                    elif dim == 2:
+                        theta_t[talphas:talphae,:] = theta_0[key][talphas:talphae,:].clone()
+
+                    elif dim == 3:
+                        theta_t[talphas:talphae,:,:] = theta_0[key][talphas:talphae,:,:].clone()
+
+                    elif dim == 4:
+                        theta_t[talphas:talphae,:,:,:] = theta_0[key][talphas:talphae,:,:,:].clone()
+                    theta_0[key] = theta_t
+
+    currentmodel = makemodelname(weights_a,weights_b,model_a, model_b,model_c, base_alpha,base_beta,useblocks,mode)
+
+    for key in tqdm(theta_1.keys(), desc="Stage 2/2"):
+        if key in chckpoint_dict_skip_on_merge:
+            continue
+        if "model" in key and key not in theta_0:
+            theta_0.update({key:theta_1[key]})
+
+    del theta_1
+
+    modelid = rwmergelog(currentmodel,mergedmodel)
+
+    caster(mergedmodel,False)
+
+    if save_metadata:
+        merge_recipe = {
+            "type": "sd-webui-supermerger",
+            "weights_alpha": weights_a if useblocks else None,
+            "weights_beta": weights_b if useblocks else None,
+            "weights_alpha_orig": weights_a_orig if useblocks else None,
+            "weights_beta_orig": weights_b_orig if useblocks else None,
+            "model_a": longhashfromname(model_a),
+            "model_b": longhashfromname(model_b),
+            "model_c": longhashfromname(model_c),
+            "base_alpha": base_alpha,
+            "base_beta": base_beta,
+            "mode": mode,
+            "mbw": useblocks,
+            "elemental_merge": deep,
+            "calcmode" : calcmode
+        }
+        metadata["sd_merge_recipe"] = json.dumps(merge_recipe)
+        metadata["sd_merge_models"] = {}
+
+        def add_model_metadata(checkpoint_name):
+            checkpoint_info = sd_models.get_closet_checkpoint_match(checkpoint_name)
+            checkpoint_info.calculate_shorthash()
+            metadata["sd_merge_models"][checkpoint_info.sha256] = {
+                "name": checkpoint_name,
+                "legacy_hash": checkpoint_info.hash
+            }
+
+            #metadata["sd_merge_models"].update(checkpoint_info.metadata.get("sd_merge_models", {}))
+
+        if model_a:
+            add_model_metadata(model_a)
+        if model_b:
+            add_model_metadata(model_b)
+        if model_c:
+            add_model_metadata(model_c)
+
+        metadata["sd_merge_models"] = json.dumps(metadata["sd_merge_models"])
+
+    return "",currentmodel,modelid,theta_0,metadata
+def forkforker(filename):
+    try:
+        return sd_models.read_state_dict(filename,"cuda")
+    except:
+        return sd_models.read_state_dict(filename)
+
+def load_model_weights_m(model,model_a,model_b,save):
+    checkpoint_info = sd_models.get_closet_checkpoint_match(model)
+    sd_model_name = checkpoint_info.model_name
+
+    cachenum = shared.opts.sd_checkpoint_cache
+    
+    if save:        
+        if model_a:
+            load_model(checkpoint_info)
+        print(f"Loading weights [{sd_model_name}] from file")
+        return forkforker(checkpoint_info.filename)
+
+    if checkpoint_info in checkpoints_loaded:
+        print(f"Loading weights [{sd_model_name}] from cache")
+        return checkpoints_loaded[checkpoint_info]
+    elif cachenum>0 and model_a:
+        load_model(checkpoint_info)
+        print(f"Loading weights [{sd_model_name}] from cache")
+        return checkpoints_loaded[checkpoint_info]
+    elif cachenum>1 and model_b:
+        load_model(checkpoint_info)
+        print(f"Loading weights [{sd_model_name}] from cache")
+        return checkpoints_loaded[checkpoint_info]
+    elif cachenum>2:
+        load_model(checkpoint_info)
+        print(f"Loading weights [{sd_model_name}] from cache")
+        return checkpoints_loaded[checkpoint_info]
+    else:
+        if model_a:
+            load_model(checkpoint_info)
+        print(f"Loading weights [{sd_model_name}] from file")
+        return forkforker(checkpoint_info.filename)
+
+def makemodelname(weights_a,weights_b,model_a, model_b,model_c, alpha,beta,useblocks,mode):
+    model_a=filenamecutter(model_a)
+    model_b=filenamecutter(model_b)
+    model_c=filenamecutter(model_c)
+
+    if type(alpha) == str:alpha = float(alpha)
+    if type(beta)== str:beta  = float(beta)
+
+    if useblocks:
+        if MODES[1] in mode:#add
+            currentmodel =f"{model_a} + ({model_b} - {model_c}) x alpha ({str(round(alpha,3))},{','.join(str(s) for s in weights_a)}"
+        elif MODES[2] in mode:#triple
+            currentmodel =f"{model_a} x (1-alpha-beta) + {model_b} x alpha + {model_c} x beta (alpha = {str(round(alpha,3))},{','.join(str(s) for s in weights_a)},beta = {beta},{','.join(str(s) for s in weights_b)})"
+        elif MODES[3] in mode:#twice
+            currentmodel =f"({model_a} x (1-alpha) + {model_b} x alpha)x(1-beta)+  {model_c} x beta ({str(round(alpha,3))},{','.join(str(s) for s in weights_a)})_({str(round(beta,3))},{','.join(str(s) for s in weights_b)})"
+        else:
+            currentmodel =f"{model_a} x (1-alpha) + {model_b} x alpha ({str(round(alpha,3))},{','.join(str(s) for s in weights_a)})"
+    else:
+        if MODES[1] in mode:#add
+            currentmodel =f"{model_a} + ({model_b} -  {model_c}) x {str(round(alpha,3))}"
+        elif MODES[2] in mode:#triple
+            currentmodel =f"{model_a} x {str(round(1-alpha-beta,3))} + {model_b} x {str(round(alpha,3))} + {model_c} x {str(round(beta,3))}"
+        elif MODES[3] in mode:#twice
+            currentmodel =f"({model_a} x {str(round(1-alpha,3))} +{model_b} x {str(round(alpha,3))}) x {str(round(1-beta,3))} + {model_c} x {str(round(beta,3))}"
+        else:
+            currentmodel =f"{model_a} x {str(round(1-alpha,3))} + {model_b} x {str(round(alpha,3))}"
+    return currentmodel
+
+path_root = scripts.basedir()
+
+def rwmergelog(mergedname = "",settings= [],id = 0):
+    setting = settings.copy()
+    filepath = os.path.join(path_root, "mergehistory.csv")
+    is_file = os.path.isfile(filepath)
+    if not is_file:
+        with open(filepath, 'a') as f:
+                                       #msettings=[0 weights_a,1 weights_b,2 model_a,3 model_b,4 model_c,5 base_alpha,6 base_beta,7 mode,8 useblocks,9 custom_name,10 save_sets,11 id_sets, 12 deep 13 calcmode]
+            f.writelines('"ID","time","name","weights alpha","weights beta","model A","model B","model C","alpha","beta","mode","use MBW","plus lora","custum name","save setting","use ID"\n')
+    with  open(filepath, 'r+') as f:
+        reader = csv.reader(f)
+        mlist = [raw for raw in reader]
+        if mergedname != "":
+            mergeid = len(mlist)
+            setting.insert(0,mergedname)
+            for i,x in enumerate(setting):
+                if "," in str(x):setting[i] = f'"{str(setting[i])}"'
+            text = ",".join(map(str, setting))
+            text=str(mergeid)+","+datetime.datetime.now().strftime('%Y.%m.%d %H.%M.%S.%f')[:-7]+"," + text + "\n"
+            f.writelines(text)
+            return mergeid
+        try:
+            out = mlist[int(id)]
+        except:
+            out = "ERROR: OUT of ID index"
+        return out
+
+def draw_origin(grid, text,width,height,width_one):
+    grid_d= Image.new("RGB", (grid.width,grid.height), "white")
+    grid_d.paste(grid,(0,0))
+    def get_font(fontsize):
+        try:
+            from fonts.ttf import Roboto
+            try:
+                return ImageFont.truetype(opts.font or Roboto, fontsize)
+            except Exception:
+                return ImageFont.truetype(Roboto, fontsize)
+        except Exception:
+            try:
+                return ImageFont.truetype(shared.opts.font or 'javascript/roboto.ttf', fontsize)
+            except Exception:
+                return ImageFont.truetype('javascript/roboto.ttf', fontsize)
+
+    d= ImageDraw.Draw(grid_d)
+    color_active = (0, 0, 0)
+    fontsize = (width+height)//25
+    fnt = get_font(fontsize)
+
+    if grid.width != width_one:
+        while d.multiline_textsize(text, font=fnt)[0] > width_one*0.75 and fontsize > 0:
+            fontsize -=1
+            fnt = get_font(fontsize)
+    d.multiline_text((0,0), text, font=fnt, fill=color_active,align="center")
+    return grid_d
+
+def wpreseter(w,presets):
+    if "," not in w and w != "":
+        presets=presets.splitlines()
+        wdict={}
+        for l in presets:
+            if ":" in l :
+                key = l.split(":",1)[0]
+                wdict[key.strip()]=l.split(":",1)[1]
+            if "\t" in l:
+                key = l.split("\t",1)[0]
+                wdict[key.strip()]=l.split("\t",1)[1]
+        if w.strip() in wdict:
+            name = w
+            w = wdict[w.strip()]
+            print(f"weights {name} imported from presets : {w}")
+    return w
+
+def fullpathfromname(name):
+    if hash == "" or hash ==[]: return ""
+    checkpoint_info = sd_models.get_closet_checkpoint_match(name)
+    return checkpoint_info.filename
+
+def namefromhash(hash):
+    if hash == "" or hash ==[]: return ""
+    checkpoint_info = sd_models.get_closet_checkpoint_match(hash)
+    return checkpoint_info.model_name
+
+def hashfromname(name):
+    from modules import sd_models
+    if name == "" or name ==[]: return ""
+    checkpoint_info = sd_models.get_closet_checkpoint_match(name)
+    if checkpoint_info.shorthash is not None:
+        return checkpoint_info.shorthash
+    return checkpoint_info.calculate_shorthash()
+
+def longhashfromname(name):
+    from modules import sd_models
+    if name == "" or name ==[]: return ""
+    checkpoint_info = sd_models.get_closet_checkpoint_match(name)
+    if checkpoint_info.sha256 is not None:
+        return checkpoint_info.sha256
+    checkpoint_info.calculate_shorthash()
+    return checkpoint_info.sha256
+
+def simggen(prompt, nprompt, steps, sampler, cfg, seed, w, h,genoptions,hrupscaler,hr2ndsteps,denoise_str,hr_scale,batch_size,mergeinfo="",id_sets=[],modelid = "no id"):
+    shared.state.begin()
+    p = processing.StableDiffusionProcessingTxt2Img(
+        sd_model=shared.sd_model,
+        do_not_save_grid=True,
+        do_not_save_samples=True,
+        do_not_reload_embeddings=True,
+    )
+    p.batch_size = int(batch_size)
+    p.prompt = prompt
+    p.negative_prompt = nprompt
+    p.steps = steps
+    p.sampler_name = sd_samplers.samplers[sampler].name
+    p.cfg_scale = cfg
+    p.seed = seed
+    p.width = w
+    p.height = h
+    p.seed_resize_from_w=0
+    p.seed_resize_from_h=0
+    p.denoising_strength=None
+
+    #"Restore faces", "Tiling", "Hires. fix"
+
+    if "Hires. fix" in genoptions:
+        p.enable_hr = True
+        p.denoising_strength = denoise_str
+        p.hr_upscaler = hrupscaler
+        p.hr_second_pass_steps = hr2ndsteps
+        p.hr_scale = hr_scale
+    
+    if "Tiling" in genoptions:
+        p.tiling = True
+
+    if "Restore faces" in genoptions:
+        p.restore_faces = True
+
+    if type(p.prompt) == list:
+        p.all_prompts = [shared.prompt_styles.apply_styles_to_prompt(x, p.styles) for x in p.prompt]
+    else:
+        p.all_prompts = [shared.prompt_styles.apply_styles_to_prompt(p.prompt, p.styles)]
+
+    if type(p.negative_prompt) == list:
+        p.all_negative_prompts = [shared.prompt_styles.apply_negative_styles_to_prompt(x, p.styles) for x in p.negative_prompt]
+    else:
+        p.all_negative_prompts = [shared.prompt_styles.apply_negative_styles_to_prompt(p.negative_prompt, p.styles)]
+
+    processed:Processed = processing.process_images(p)
+    if "image" in id_sets:
+        for i, image in enumerate(processed.images):
+            processed.images[i] = draw_origin(image, str(modelid),w,h,w)
+
+    if "PNG info" in id_sets:mergeinfo = mergeinfo + " ID " + str(modelid)
+
+    infotext = create_infotext(p, p.all_prompts, p.all_seeds, p.all_subseeds)
+    if infotext.count("Steps: ")>1:
+        infotext = infotext[:infotext.rindex("Steps")]
+
+    infotexts = infotext.split(",")
+    for i,x in enumerate(infotexts):
+        if "Model:"in x:
+            infotexts[i] = " Model: "+mergeinfo.replace(","," ")
+    infotext= ",".join(infotexts)
+
+    for i, image in enumerate(processed.images):
+        images.save_image(image, opts.outdir_txt2img_samples, "",p.seed, p.prompt,shared.opts.samples_format, p=p,info=infotext)
+
+    if batch_size > 1:
+        grid = images.image_grid(processed.images, p.batch_size)
+        processed.images.insert(0, grid)
+        images.save_image(grid, opts.outdir_txt2img_grids, "grid", p.seed, p.prompt, opts.grid_format, info=infotext, short_filename=not opts.grid_extended_filename, p=p, grid=True)
+    shared.state.end()
+    return processed.images,infotext,plaintext_to_html(processed.info), plaintext_to_html(processed.comments),p

microsoftexcel-supermerger/scripts/mergers/model_util.py

@@ -0,0 +1,928 @@
+import os
+import torch
+from transformers import CLIPTextModel,  CLIPTextConfig
+from safetensors.torch import load_file
+import safetensors.torch
+from modules.sd_models import read_state_dict
+
+# DiffUsers版StableDiffusionのモデルパラメータ
+NUM_TRAIN_TIMESTEPS = 1000
+BETA_START = 0.00085
+BETA_END = 0.0120
+
+UNET_PARAMS_MODEL_CHANNELS = 320
+UNET_PARAMS_CHANNEL_MULT = [1, 2, 4, 4]
+UNET_PARAMS_ATTENTION_RESOLUTIONS = [4, 2, 1]
+UNET_PARAMS_IMAGE_SIZE = 64  # fixed from old invalid value `32`
+UNET_PARAMS_IN_CHANNELS = 4
+UNET_PARAMS_OUT_CHANNELS = 4
+UNET_PARAMS_NUM_RES_BLOCKS = 2
+UNET_PARAMS_CONTEXT_DIM = 768
+UNET_PARAMS_NUM_HEADS = 8
+
+VAE_PARAMS_Z_CHANNELS = 4
+VAE_PARAMS_RESOLUTION = 256
+VAE_PARAMS_IN_CHANNELS = 3
+VAE_PARAMS_OUT_CH = 3
+VAE_PARAMS_CH = 128
+VAE_PARAMS_CH_MULT = [1, 2, 4, 4]
+VAE_PARAMS_NUM_RES_BLOCKS = 2
+
+# V2
+V2_UNET_PARAMS_ATTENTION_HEAD_DIM = [5, 10, 20, 20]
+V2_UNET_PARAMS_CONTEXT_DIM = 1024
+
+# Diffusersの設定を読み込むための参照モデル
+DIFFUSERS_REF_MODEL_ID_V1 = "runwayml/stable-diffusion-v1-5"
+DIFFUSERS_REF_MODEL_ID_V2 = "stabilityai/stable-diffusion-2-1"
+
+
+# region StableDiffusion->Diffusersの変換コード
+# convert_original_stable_diffusion_to_diffusers をコピーして修正している（ASL 2.0）
+
+
+def shave_segments(path, n_shave_prefix_segments=1):
+  """
+  Removes segments. Positive values shave the first segments, negative shave the last segments.
+  """
+  if n_shave_prefix_segments >= 0:
+    return ".".join(path.split(".")[n_shave_prefix_segments:])
+  else:
+    return ".".join(path.split(".")[:n_shave_prefix_segments])
+
+
+def renew_resnet_paths(old_list, n_shave_prefix_segments=0):
+  """
+  Updates paths inside resnets to the new naming scheme (local renaming)
+  """
+  mapping = []
+  for old_item in old_list:
+    new_item = old_item.replace("in_layers.0", "norm1")
+    new_item = new_item.replace("in_layers.2", "conv1")
+
+    new_item = new_item.replace("out_layers.0", "norm2")
+    new_item = new_item.replace("out_layers.3", "conv2")
+
+    new_item = new_item.replace("emb_layers.1", "time_emb_proj")
+    new_item = new_item.replace("skip_connection", "conv_shortcut")
+
+    new_item = shave_segments(new_item, n_shave_prefix_segments=n_shave_prefix_segments)
+
+    mapping.append({"old": old_item, "new": new_item})
+
+  return mapping
+
+
+def renew_vae_resnet_paths(old_list, n_shave_prefix_segments=0):
+  """
+  Updates paths inside resnets to the new naming scheme (local renaming)
+  """
+  mapping = []
+  for old_item in old_list:
+    new_item = old_item
+
+    new_item = new_item.replace("nin_shortcut", "conv_shortcut")
+    new_item = shave_segments(new_item, n_shave_prefix_segments=n_shave_prefix_segments)
+
+    mapping.append({"old": old_item, "new": new_item})
+
+  return mapping
+
+
+def renew_attention_paths(old_list, n_shave_prefix_segments=0):
+  """
+  Updates paths inside attentions to the new naming scheme (local renaming)
+  """
+  mapping = []
+  for old_item in old_list:
+    new_item = old_item
+
+    #         new_item = new_item.replace('norm.weight', 'group_norm.weight')
+    #         new_item = new_item.replace('norm.bias', 'group_norm.bias')
+
+    #         new_item = new_item.replace('proj_out.weight', 'proj_attn.weight')
+    #         new_item = new_item.replace('proj_out.bias', 'proj_attn.bias')
+
+    #         new_item = shave_segments(new_item, n_shave_prefix_segments=n_shave_prefix_segments)
+
+    mapping.append({"old": old_item, "new": new_item})
+
+  return mapping
+
+
+def renew_vae_attention_paths(old_list, n_shave_prefix_segments=0):
+  """
+  Updates paths inside attentions to the new naming scheme (local renaming)
+  """
+  mapping = []
+  for old_item in old_list:
+    new_item = old_item
+
+    new_item = new_item.replace("norm.weight", "group_norm.weight")
+    new_item = new_item.replace("norm.bias", "group_norm.bias")
+
+    new_item = new_item.replace("q.weight", "query.weight")
+    new_item = new_item.replace("q.bias", "query.bias")
+
+    new_item = new_item.replace("k.weight", "key.weight")
+    new_item = new_item.replace("k.bias", "key.bias")
+
+    new_item = new_item.replace("v.weight", "value.weight")
+    new_item = new_item.replace("v.bias", "value.bias")
+
+    new_item = new_item.replace("proj_out.weight", "proj_attn.weight")
+    new_item = new_item.replace("proj_out.bias", "proj_attn.bias")
+
+    new_item = shave_segments(new_item, n_shave_prefix_segments=n_shave_prefix_segments)
+
+    mapping.append({"old": old_item, "new": new_item})
+
+  return mapping
+
+
+def assign_to_checkpoint(
+    paths, checkpoint, old_checkpoint, attention_paths_to_split=None, additional_replacements=None, config=None
+):
+  """
+  This does the final conversion step: take locally converted weights and apply a global renaming
+  to them. It splits attention layers, and takes into account additional replacements
+  that may arise.
+
+  Assigns the weights to the new checkpoint.
+  """
+  assert isinstance(paths, list), "Paths should be a list of dicts containing 'old' and 'new' keys."
+
+  # Splits the attention layers into three variables.
+  if attention_paths_to_split is not None:
+    for path, path_map in attention_paths_to_split.items():
+      old_tensor = old_checkpoint[path]
+      channels = old_tensor.shape[0] // 3
+
+      target_shape = (-1, channels) if len(old_tensor.shape) == 3 else (-1)
+
+      num_heads = old_tensor.shape[0] // config["num_head_channels"] // 3
+
+      old_tensor = old_tensor.reshape((num_heads, 3 * channels // num_heads) + old_tensor.shape[1:])
+      query, key, value = old_tensor.split(channels // num_heads, dim=1)
+
+      checkpoint[path_map["query"]] = query.reshape(target_shape)
+      checkpoint[path_map["key"]] = key.reshape(target_shape)
+      checkpoint[path_map["value"]] = value.reshape(target_shape)
+
+  for path in paths:
+    new_path = path["new"]
+
+    # These have already been assigned
+    if attention_paths_to_split is not None and new_path in attention_paths_to_split:
+      continue
+
+    # Global renaming happens here
+    new_path = new_path.replace("middle_block.0", "mid_block.resnets.0")
+    new_path = new_path.replace("middle_block.1", "mid_block.attentions.0")
+    new_path = new_path.replace("middle_block.2", "mid_block.resnets.1")
+
+    if additional_replacements is not None:
+      for replacement in additional_replacements:
+        new_path = new_path.replace(replacement["old"], replacement["new"])
+
+    # proj_attn.weight has to be converted from conv 1D to linear
+    if "proj_attn.weight" in new_path:
+      checkpoint[new_path] = old_checkpoint[path["old"]][:, :, 0]
+    else:
+      checkpoint[new_path] = old_checkpoint[path["old"]]
+
+
+def conv_attn_to_linear(checkpoint):
+  keys = list(checkpoint.keys())
+  attn_keys = ["query.weight", "key.weight", "value.weight"]
+  for key in keys:
+    if ".".join(key.split(".")[-2:]) in attn_keys:
+      if checkpoint[key].ndim > 2:
+        checkpoint[key] = checkpoint[key][:, :, 0, 0]
+    elif "proj_attn.weight" in key:
+      if checkpoint[key].ndim > 2:
+        checkpoint[key] = checkpoint[key][:, :, 0]
+
+
+def linear_transformer_to_conv(checkpoint):
+  keys = list(checkpoint.keys())
+  tf_keys = ["proj_in.weight", "proj_out.weight"]
+  for key in keys:
+    if ".".join(key.split(".")[-2:]) in tf_keys:
+      if checkpoint[key].ndim == 2:
+        checkpoint[key] = checkpoint[key].unsqueeze(2).unsqueeze(2)
+
+
+def convert_ldm_unet_checkpoint(v2, checkpoint, config):
+  """
+  Takes a state dict and a config, and returns a converted checkpoint.
+  """
+
+  # extract state_dict for UNet
+  unet_state_dict = {}
+  unet_key = "model.diffusion_model."
+  keys = list(checkpoint.keys())
+  for key in keys:
+    if key.startswith(unet_key):
+      unet_state_dict[key.replace(unet_key, "")] = checkpoint.pop(key)
+
+  new_checkpoint = {}
+
+  new_checkpoint["time_embedding.linear_1.weight"] = unet_state_dict["time_embed.0.weight"]
+  new_checkpoint["time_embedding.linear_1.bias"] = unet_state_dict["time_embed.0.bias"]
+  new_checkpoint["time_embedding.linear_2.weight"] = unet_state_dict["time_embed.2.weight"]
+  new_checkpoint["time_embedding.linear_2.bias"] = unet_state_dict["time_embed.2.bias"]
+
+  new_checkpoint["conv_in.weight"] = unet_state_dict["input_blocks.0.0.weight"]
+  new_checkpoint["conv_in.bias"] = unet_state_dict["input_blocks.0.0.bias"]
+
+  new_checkpoint["conv_norm_out.weight"] = unet_state_dict["out.0.weight"]
+  new_checkpoint["conv_norm_out.bias"] = unet_state_dict["out.0.bias"]
+  new_checkpoint["conv_out.weight"] = unet_state_dict["out.2.weight"]
+  new_checkpoint["conv_out.bias"] = unet_state_dict["out.2.bias"]
+
+  # Retrieves the keys for the input blocks only
+  num_input_blocks = len({".".join(layer.split(".")[:2]) for layer in unet_state_dict if "input_blocks" in layer})
+  input_blocks = {
+      layer_id: [key for key in unet_state_dict if f"input_blocks.{layer_id}." in key]
+      for layer_id in range(num_input_blocks)
+  }
+
+  # Retrieves the keys for the middle blocks only
+  num_middle_blocks = len({".".join(layer.split(".")[:2]) for layer in unet_state_dict if "middle_block" in layer})
+  middle_blocks = {
+      layer_id: [key for key in unet_state_dict if f"middle_block.{layer_id}." in key]
+      for layer_id in range(num_middle_blocks)
+  }
+
+  # Retrieves the keys for the output blocks only
+  num_output_blocks = len({".".join(layer.split(".")[:2]) for layer in unet_state_dict if "output_blocks" in layer})
+  output_blocks = {
+      layer_id: [key for key in unet_state_dict if f"output_blocks.{layer_id}." in key]
+      for layer_id in range(num_output_blocks)
+  }
+
+  for i in range(1, num_input_blocks):
+    block_id = (i - 1) // (config["layers_per_block"] + 1)
+    layer_in_block_id = (i - 1) % (config["layers_per_block"] + 1)
+
+    resnets = [
+        key for key in input_blocks[i] if f"input_blocks.{i}.0" in key and f"input_blocks.{i}.0.op" not in key
+    ]
+    attentions = [key for key in input_blocks[i] if f"input_blocks.{i}.1" in key]
+
+    if f"input_blocks.{i}.0.op.weight" in unet_state_dict:
+      new_checkpoint[f"down_blocks.{block_id}.downsamplers.0.conv.weight"] = unet_state_dict.pop(
+          f"input_blocks.{i}.0.op.weight"
+      )
+      new_checkpoint[f"down_blocks.{block_id}.downsamplers.0.conv.bias"] = unet_state_dict.pop(
+          f"input_blocks.{i}.0.op.bias"
+      )
+
+    paths = renew_resnet_paths(resnets)
+    meta_path = {"old": f"input_blocks.{i}.0", "new": f"down_blocks.{block_id}.resnets.{layer_in_block_id}"}
+    assign_to_checkpoint(
+        paths, new_checkpoint, unet_state_dict, additional_replacements=[meta_path], config=config
+    )
+
+    if len(attentions):
+      paths = renew_attention_paths(attentions)
+      meta_path = {"old": f"input_blocks.{i}.1", "new": f"down_blocks.{block_id}.attentions.{layer_in_block_id}"}
+      assign_to_checkpoint(
+          paths, new_checkpoint, unet_state_dict, additional_replacements=[meta_path], config=config
+      )
+
+  resnet_0 = middle_blocks[0]
+  attentions = middle_blocks[1]
+  resnet_1 = middle_blocks[2]
+
+  resnet_0_paths = renew_resnet_paths(resnet_0)
+  assign_to_checkpoint(resnet_0_paths, new_checkpoint, unet_state_dict, config=config)
+
+  resnet_1_paths = renew_resnet_paths(resnet_1)
+  assign_to_checkpoint(resnet_1_paths, new_checkpoint, unet_state_dict, config=config)
+
+  attentions_paths = renew_attention_paths(attentions)
+  meta_path = {"old": "middle_block.1", "new": "mid_block.attentions.0"}
+  assign_to_checkpoint(
+      attentions_paths, new_checkpoint, unet_state_dict, additional_replacements=[meta_path], config=config
+  )
+
+  for i in range(num_output_blocks):
+    block_id = i // (config["layers_per_block"] + 1)
+    layer_in_block_id = i % (config["layers_per_block"] + 1)
+    output_block_layers = [shave_segments(name, 2) for name in output_blocks[i]]
+    output_block_list = {}
+
+    for layer in output_block_layers:
+      layer_id, layer_name = layer.split(".")[0], shave_segments(layer, 1)
+      if layer_id in output_block_list:
+        output_block_list[layer_id].append(layer_name)
+      else:
+        output_block_list[layer_id] = [layer_name]
+
+    if len(output_block_list) > 1:
+      resnets = [key for key in output_blocks[i] if f"output_blocks.{i}.0" in key]
+      attentions = [key for key in output_blocks[i] if f"output_blocks.{i}.1" in key]
+
+      resnet_0_paths = renew_resnet_paths(resnets)
+      paths = renew_resnet_paths(resnets)
+
+      meta_path = {"old": f"output_blocks.{i}.0", "new": f"up_blocks.{block_id}.resnets.{layer_in_block_id}"}
+      assign_to_checkpoint(
+          paths, new_checkpoint, unet_state_dict, additional_replacements=[meta_path], config=config
+      )
+
+      # オリジナル：
+      # if ["conv.weight", "conv.bias"] in output_block_list.values():
+      #   index = list(output_block_list.values()).index(["conv.weight", "conv.bias"])
+
+      # biasとweightの順番に依存しないようにする：もっといいやり方がありそうだが
+      for l in output_block_list.values():
+        l.sort()
+
+      if ["conv.bias", "conv.weight"] in output_block_list.values():
+        index = list(output_block_list.values()).index(["conv.bias", "conv.weight"])
+        new_checkpoint[f"up_blocks.{block_id}.upsamplers.0.conv.bias"] = unet_state_dict[
+            f"output_blocks.{i}.{index}.conv.bias"
+        ]
+        new_checkpoint[f"up_blocks.{block_id}.upsamplers.0.conv.weight"] = unet_state_dict[
+            f"output_blocks.{i}.{index}.conv.weight"
+        ]
+
+        # Clear attentions as they have been attributed above.
+        if len(attentions) == 2:
+          attentions = []
+
+      if len(attentions):
+        paths = renew_attention_paths(attentions)
+        meta_path = {
+            "old": f"output_blocks.{i}.1",
+            "new": f"up_blocks.{block_id}.attentions.{layer_in_block_id}",
+        }
+        assign_to_checkpoint(
+            paths, new_checkpoint, unet_state_dict, additional_replacements=[meta_path], config=config
+        )
+    else:
+      resnet_0_paths = renew_resnet_paths(output_block_layers, n_shave_prefix_segments=1)
+      for path in resnet_0_paths:
+        old_path = ".".join(["output_blocks", str(i), path["old"]])
+        new_path = ".".join(["up_blocks", str(block_id), "resnets", str(layer_in_block_id), path["new"]])
+
+        new_checkpoint[new_path] = unet_state_dict[old_path]
+
+  # SDのv2では1*1のconv2dがlinearに変わっているので、linear->convに変換する
+  if v2:
+    linear_transformer_to_conv(new_checkpoint)
+
+  return new_checkpoint
+
+
+def convert_ldm_vae_checkpoint(checkpoint, config):
+  # extract state dict for VAE
+  vae_state_dict = {}
+  vae_key = "first_stage_model."
+  keys = list(checkpoint.keys())
+  for key in keys:
+    if key.startswith(vae_key):
+      vae_state_dict[key.replace(vae_key, "")] = checkpoint.get(key)
+  # if len(vae_state_dict) == 0:
+  #   # 渡されたcheckpointは.ckptから読み込んだcheckpointではなくvaeのstate_dict
+  #   vae_state_dict = checkpoint
+
+  new_checkpoint = {}
+
+  new_checkpoint["encoder.conv_in.weight"] = vae_state_dict["encoder.conv_in.weight"]
+  new_checkpoint["encoder.conv_in.bias"] = vae_state_dict["encoder.conv_in.bias"]
+  new_checkpoint["encoder.conv_out.weight"] = vae_state_dict["encoder.conv_out.weight"]
+  new_checkpoint["encoder.conv_out.bias"] = vae_state_dict["encoder.conv_out.bias"]
+  new_checkpoint["encoder.conv_norm_out.weight"] = vae_state_dict["encoder.norm_out.weight"]
+  new_checkpoint["encoder.conv_norm_out.bias"] = vae_state_dict["encoder.norm_out.bias"]
+
+  new_checkpoint["decoder.conv_in.weight"] = vae_state_dict["decoder.conv_in.weight"]
+  new_checkpoint["decoder.conv_in.bias"] = vae_state_dict["decoder.conv_in.bias"]
+  new_checkpoint["decoder.conv_out.weight"] = vae_state_dict["decoder.conv_out.weight"]
+  new_checkpoint["decoder.conv_out.bias"] = vae_state_dict["decoder.conv_out.bias"]
+  new_checkpoint["decoder.conv_norm_out.weight"] = vae_state_dict["decoder.norm_out.weight"]
+  new_checkpoint["decoder.conv_norm_out.bias"] = vae_state_dict["decoder.norm_out.bias"]
+
+  new_checkpoint["quant_conv.weight"] = vae_state_dict["quant_conv.weight"]
+  new_checkpoint["quant_conv.bias"] = vae_state_dict["quant_conv.bias"]
+  new_checkpoint["post_quant_conv.weight"] = vae_state_dict["post_quant_conv.weight"]
+  new_checkpoint["post_quant_conv.bias"] = vae_state_dict["post_quant_conv.bias"]
+
+  # Retrieves the keys for the encoder down blocks only
+  num_down_blocks = len({".".join(layer.split(".")[:3]) for layer in vae_state_dict if "encoder.down" in layer})
+  down_blocks = {
+      layer_id: [key for key in vae_state_dict if f"down.{layer_id}" in key] for layer_id in range(num_down_blocks)
+  }
+
+  # Retrieves the keys for the decoder up blocks only
+  num_up_blocks = len({".".join(layer.split(".")[:3]) for layer in vae_state_dict if "decoder.up" in layer})
+  up_blocks = {
+      layer_id: [key for key in vae_state_dict if f"up.{layer_id}" in key] for layer_id in range(num_up_blocks)
+  }
+
+  for i in range(num_down_blocks):
+    resnets = [key for key in down_blocks[i] if f"down.{i}" in key and f"down.{i}.downsample" not in key]
+
+    if f"encoder.down.{i}.downsample.conv.weight" in vae_state_dict:
+      new_checkpoint[f"encoder.down_blocks.{i}.downsamplers.0.conv.weight"] = vae_state_dict.pop(
+          f"encoder.down.{i}.downsample.conv.weight"
+      )
+      new_checkpoint[f"encoder.down_blocks.{i}.downsamplers.0.conv.bias"] = vae_state_dict.pop(
+          f"encoder.down.{i}.downsample.conv.bias"
+      )
+
+    paths = renew_vae_resnet_paths(resnets)
+    meta_path = {"old": f"down.{i}.block", "new": f"down_blocks.{i}.resnets"}
+    assign_to_checkpoint(paths, new_checkpoint, vae_state_dict, additional_replacements=[meta_path], config=config)
+
+  mid_resnets = [key for key in vae_state_dict if "encoder.mid.block" in key]
+  num_mid_res_blocks = 2
+  for i in range(1, num_mid_res_blocks + 1):
+    resnets = [key for key in mid_resnets if f"encoder.mid.block_{i}" in key]
+
+    paths = renew_vae_resnet_paths(resnets)
+    meta_path = {"old": f"mid.block_{i}", "new": f"mid_block.resnets.{i - 1}"}
+    assign_to_checkpoint(paths, new_checkpoint, vae_state_dict, additional_replacements=[meta_path], config=config)
+
+  mid_attentions = [key for key in vae_state_dict if "encoder.mid.attn" in key]
+  paths = renew_vae_attention_paths(mid_attentions)
+  meta_path = {"old": "mid.attn_1", "new": "mid_block.attentions.0"}
+  assign_to_checkpoint(paths, new_checkpoint, vae_state_dict, additional_replacements=[meta_path], config=config)
+  conv_attn_to_linear(new_checkpoint)
+
+  for i in range(num_up_blocks):
+    block_id = num_up_blocks - 1 - i
+    resnets = [
+        key for key in up_blocks[block_id] if f"up.{block_id}" in key and f"up.{block_id}.upsample" not in key
+    ]
+
+    if f"decoder.up.{block_id}.upsample.conv.weight" in vae_state_dict:
+      new_checkpoint[f"decoder.up_blocks.{i}.upsamplers.0.conv.weight"] = vae_state_dict[
+          f"decoder.up.{block_id}.upsample.conv.weight"
+      ]
+      new_checkpoint[f"decoder.up_blocks.{i}.upsamplers.0.conv.bias"] = vae_state_dict[
+          f"decoder.up.{block_id}.upsample.conv.bias"
+      ]
+
+    paths = renew_vae_resnet_paths(resnets)
+    meta_path = {"old": f"up.{block_id}.block", "new": f"up_blocks.{i}.resnets"}
+    assign_to_checkpoint(paths, new_checkpoint, vae_state_dict, additional_replacements=[meta_path], config=config)
+
+  mid_resnets = [key for key in vae_state_dict if "decoder.mid.block" in key]
+  num_mid_res_blocks = 2
+  for i in range(1, num_mid_res_blocks + 1):
+    resnets = [key for key in mid_resnets if f"decoder.mid.block_{i}" in key]
+
+    paths = renew_vae_resnet_paths(resnets)
+    meta_path = {"old": f"mid.block_{i}", "new": f"mid_block.resnets.{i - 1}"}
+    assign_to_checkpoint(paths, new_checkpoint, vae_state_dict, additional_replacements=[meta_path], config=config)
+
+  mid_attentions = [key for key in vae_state_dict if "decoder.mid.attn" in key]
+  paths = renew_vae_attention_paths(mid_attentions)
+  meta_path = {"old": "mid.attn_1", "new": "mid_block.attentions.0"}
+  assign_to_checkpoint(paths, new_checkpoint, vae_state_dict, additional_replacements=[meta_path], config=config)
+  conv_attn_to_linear(new_checkpoint)
+  return new_checkpoint
+
+
+def create_unet_diffusers_config(v2):
+  """
+  Creates a config for the diffusers based on the config of the LDM model.
+  """
+  # unet_params = original_config.model.params.unet_config.params
+
+  block_out_channels = [UNET_PARAMS_MODEL_CHANNELS * mult for mult in UNET_PARAMS_CHANNEL_MULT]
+
+  down_block_types = []
+  resolution = 1
+  for i in range(len(block_out_channels)):
+    block_type = "CrossAttnDownBlock2D" if resolution in UNET_PARAMS_ATTENTION_RESOLUTIONS else "DownBlock2D"
+    down_block_types.append(block_type)
+    if i != len(block_out_channels) - 1:
+      resolution *= 2
+
+  up_block_types = []
+  for i in range(len(block_out_channels)):
+    block_type = "CrossAttnUpBlock2D" if resolution in UNET_PARAMS_ATTENTION_RESOLUTIONS else "UpBlock2D"
+    up_block_types.append(block_type)
+    resolution //= 2
+
+  config = dict(
+      sample_size=UNET_PARAMS_IMAGE_SIZE,
+      in_channels=UNET_PARAMS_IN_CHANNELS,
+      out_channels=UNET_PARAMS_OUT_CHANNELS,
+      down_block_types=tuple(down_block_types),
+      up_block_types=tuple(up_block_types),
+      block_out_channels=tuple(block_out_channels),
+      layers_per_block=UNET_PARAMS_NUM_RES_BLOCKS,
+      cross_attention_dim=UNET_PARAMS_CONTEXT_DIM if not v2 else V2_UNET_PARAMS_CONTEXT_DIM,
+      attention_head_dim=UNET_PARAMS_NUM_HEADS if not v2 else V2_UNET_PARAMS_ATTENTION_HEAD_DIM,
+  )
+
+  return config
+
+
+def create_vae_diffusers_config():
+  """
+  Creates a config for the diffusers based on the config of the LDM model.
+  """
+  # vae_params = original_config.model.params.first_stage_config.params.ddconfig
+  # _ = original_config.model.params.first_stage_config.params.embed_dim
+  block_out_channels = [VAE_PARAMS_CH * mult for mult in VAE_PARAMS_CH_MULT]
+  down_block_types = ["DownEncoderBlock2D"] * len(block_out_channels)
+  up_block_types = ["UpDecoderBlock2D"] * len(block_out_channels)
+
+  config = dict(
+      sample_size=VAE_PARAMS_RESOLUTION,
+      in_channels=VAE_PARAMS_IN_CHANNELS,
+      out_channels=VAE_PARAMS_OUT_CH,
+      down_block_types=tuple(down_block_types),
+      up_block_types=tuple(up_block_types),
+      block_out_channels=tuple(block_out_channels),
+      latent_channels=VAE_PARAMS_Z_CHANNELS,
+      layers_per_block=VAE_PARAMS_NUM_RES_BLOCKS,
+  )
+  return config
+
+
+def convert_ldm_clip_checkpoint_v1(checkpoint):
+  keys = list(checkpoint.keys())
+  text_model_dict = {}
+  for key in keys:
+    if key.startswith("cond_stage_model.transformer"):
+      text_model_dict[key[len("cond_stage_model.transformer."):]] = checkpoint[key]
+  return text_model_dict
+
+
+def convert_ldm_clip_checkpoint_v2(checkpoint, max_length):
+  # 嫌になるくらい違うぞ！
+  def convert_key(key):
+    if not key.startswith("cond_stage_model"):
+      return None
+
+    # common conversion
+    key = key.replace("cond_stage_model.model.transformer.", "text_model.encoder.")
+    key = key.replace("cond_stage_model.model.", "text_model.")
+
+    if "resblocks" in key:
+      # resblocks conversion
+      key = key.replace(".resblocks.", ".layers.")
+      if ".ln_" in key:
+        key = key.replace(".ln_", ".layer_norm")
+      elif ".mlp." in key:
+        key = key.replace(".c_fc.", ".fc1.")
+        key = key.replace(".c_proj.", ".fc2.")
+      elif '.attn.out_proj' in key:
+        key = key.replace(".attn.out_proj.", ".self_attn.out_proj.")
+      elif '.attn.in_proj' in key:
+        key = None                  # 特殊なので後で処理する
+      else:
+        raise ValueError(f"unexpected key in SD: {key}")
+    elif '.positional_embedding' in key:
+      key = key.replace(".positional_embedding", ".embeddings.position_embedding.weight")
+    elif '.text_projection' in key:
+      key = None    # 使われない???
+    elif '.logit_scale' in key:
+      key = None    # 使われない???
+    elif '.token_embedding' in key:
+      key = key.replace(".token_embedding.weight", ".embeddings.token_embedding.weight")
+    elif '.ln_final' in key:
+      key = key.replace(".ln_final", ".final_layer_norm")
+    return key
+
+  keys = list(checkpoint.keys())
+  new_sd = {}
+  for key in keys:
+    # remove resblocks 23
+    if '.resblocks.23.' in key:
+      continue
+    new_key = convert_key(key)
+    if new_key is None:
+      continue
+    new_sd[new_key] = checkpoint[key]
+
+  # attnの変換
+  for key in keys:
+    if '.resblocks.23.' in key:
+      continue
+    if '.resblocks' in key and '.attn.in_proj_' in key:
+      # 三つに分割
+      values = torch.chunk(checkpoint[key], 3)
+
+      key_suffix = ".weight" if "weight" in key else ".bias"
+      key_pfx = key.replace("cond_stage_model.model.transformer.resblocks.", "text_model.encoder.layers.")
+      key_pfx = key_pfx.replace("_weight", "")
+      key_pfx = key_pfx.replace("_bias", "")
+      key_pfx = key_pfx.replace(".attn.in_proj", ".self_attn.")
+      new_sd[key_pfx + "q_proj" + key_suffix] = values[0]
+      new_sd[key_pfx + "k_proj" + key_suffix] = values[1]
+      new_sd[key_pfx + "v_proj" + key_suffix] = values[2]
+
+  # rename or add position_ids
+  ANOTHER_POSITION_IDS_KEY = "text_model.encoder.text_model.embeddings.position_ids"
+  if ANOTHER_POSITION_IDS_KEY in new_sd:
+    # waifu diffusion v1.4
+    position_ids = new_sd[ANOTHER_POSITION_IDS_KEY]
+    del new_sd[ANOTHER_POSITION_IDS_KEY]
+  else:
+    position_ids = torch.Tensor([list(range(max_length))]).to(torch.int64)
+
+  new_sd["text_model.embeddings.position_ids"] = position_ids
+  return new_sd
+
+def is_safetensors(path):
+  return os.path.splitext(path)[1].lower() == '.safetensors'
+
+def load_checkpoint_with_text_encoder_conversion(ckpt_path):
+  # text encoderの格納形式が違うモデルに対応する ('text_model'がない)
+  TEXT_ENCODER_KEY_REPLACEMENTS = [
+      ('cond_stage_model.transformer.embeddings.', 'cond_stage_model.transformer.text_model.embeddings.'),
+      ('cond_stage_model.transformer.encoder.', 'cond_stage_model.transformer.text_model.encoder.'),
+      ('cond_stage_model.transformer.final_layer_norm.', 'cond_stage_model.transformer.text_model.final_layer_norm.')
+  ]
+
+  state_dict = read_state_dict(ckpt_path)
+
+  key_reps = []
+  for rep_from, rep_to in TEXT_ENCODER_KEY_REPLACEMENTS:
+    for key in state_dict.keys():
+      if key.startswith(rep_from):
+        new_key = rep_to + key[len(rep_from):]
+        key_reps.append((key, new_key))
+
+  for key, new_key in key_reps:
+    state_dict[new_key] = state_dict[key]
+    del state_dict[key]
+
+  return state_dict
+
+def to_half(sd):
+    for key in sd.keys():
+        if 'model' in key and sd[key].dtype == torch.float:
+            sd[key] = sd[key].half()
+    return sd
+
+def savemodel(state_dict,currentmodel,fname,savesets,model_a,metadata={}):
+    from modules import sd_models,shared
+    if "fp16" in savesets: 
+        state_dict = to_half(state_dict)
+        pre = "fp16"
+    else:pre = ""
+    ext = ".safetensors" if "safetensors" in savesets else ".ckpt"
+
+    checkpoint_info = sd_models.get_closet_checkpoint_match(model_a)
+    model_a_path= checkpoint_info.filename
+    modeldir = os.path.split(model_a_path)[0]
+
+    if not fname or fname == "":
+        fname = currentmodel.replace(" ","").replace(",","_").replace("(","_").replace(")","_")+pre+ext
+        if fname[0]=="_":fname = fname[1:]
+    else:
+        fname = fname if ext in fname else fname +pre+ext
+
+    fname = os.path.join(modeldir, fname)
+
+    if len(fname) > 255:
+       fname.replace(ext,"")
+       fname=fname[:240]+ext
+
+    # check if output file already exists
+    if os.path.isfile(fname) and not "overwrite" in savesets:
+        _err_msg = f"Output file ({fname}) existed and was not saved]"
+        print(_err_msg)
+        return _err_msg
+
+    print("Saving...")
+    if ext == ".safetensors":
+        safetensors.torch.save_file(state_dict, fname, metadata=metadata)
+    else:
+        torch.save(state_dict, fname)
+    print("Done!")
+    return "Merged model saved in "+fname
+
+def filenamecutter(name,model_a = False):
+    from modules import sd_models
+    if name =="" or name ==[]: return
+    checkpoint_info = sd_models.get_closet_checkpoint_match(name)
+    name= os.path.splitext(checkpoint_info.filename)[0]
+
+    if not model_a:
+        name = os.path.basename(name)
+    return name
+
+# TODO dtype指定の動作が怪しいので確認する text_encoderを指定形式で作れるか未確認
+def load_models_from_stable_diffusion_checkpoint(v2, ckpt_path, dtype=None):
+  import diffusers
+  print("diffusers version : ",diffusers.__version__)
+  state_dict = load_checkpoint_with_text_encoder_conversion(ckpt_path)
+  if dtype is not None:
+    for k, v in state_dict.items():
+      if type(v) is torch.Tensor:
+        state_dict[k] = v.to(dtype)
+
+  # Convert the UNet2DConditionModel model.
+  unet_config = create_unet_diffusers_config(v2)
+  converted_unet_checkpoint = convert_ldm_unet_checkpoint(v2, state_dict, unet_config)
+
+  unet = diffusers.UNet2DConditionModel(**unet_config)
+  info = unet.load_state_dict(converted_unet_checkpoint)
+  print("loading u-net:", info)
+
+  # Convert the VAE model.
+  vae_config = create_vae_diffusers_config()
+  converted_vae_checkpoint = convert_ldm_vae_checkpoint(state_dict, vae_config)
+
+  vae = diffusers.AutoencoderKL(**vae_config)
+  info = vae.load_state_dict(converted_vae_checkpoint)
+  print("loading vae:", info)
+
+  # convert text_model
+  if v2:
+    converted_text_encoder_checkpoint = convert_ldm_clip_checkpoint_v2(state_dict, 77)
+    cfg = CLIPTextConfig(
+        vocab_size=49408,
+        hidden_size=1024,
+        intermediate_size=4096,
+        num_hidden_layers=23,
+        num_attention_heads=16,
+        max_position_embeddings=77,
+        hidden_act="gelu",
+        layer_norm_eps=1e-05,
+        dropout=0.0,
+        attention_dropout=0.0,
+        initializer_range=0.02,
+        initializer_factor=1.0,
+        pad_token_id=1,
+        bos_token_id=0,
+        eos_token_id=2,
+        model_type="clip_text_model",
+        projection_dim=512,
+        torch_dtype="float32",
+        transformers_version="4.25.0.dev0",
+    )
+    text_model = CLIPTextModel._from_config(cfg)
+    info = text_model.load_state_dict(converted_text_encoder_checkpoint)
+  else:
+    converted_text_encoder_checkpoint = convert_ldm_clip_checkpoint_v1(state_dict)
+    text_model = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14")
+    info = text_model.load_state_dict(converted_text_encoder_checkpoint)
+  print("loading text encoder:", info)
+
+  return text_model, vae, unet
+
+def usemodelgen(theta_0,model_a,model_name):
+  from modules import lowvram, devices, sd_hijack,shared, sd_vae
+  sd_hijack.model_hijack.undo_hijack(shared.sd_model)
+
+  model = shared.sd_model
+  model.load_state_dict(theta_0, strict=False)
+  del theta_0
+  if shared.cmd_opts.opt_channelslast:
+      model.to(memory_format=torch.channels_last)
+
+  if not shared.cmd_opts.no_half:
+      vae = model.first_stage_model
+
+      # with --no-half-vae, remove VAE from model when doing half() to prevent its weights from being converted to float16
+      if shared.cmd_opts.no_half_vae:
+          model.first_stage_model = None
+
+      model.half()
+      model.first_stage_model = vae
+
+  devices.dtype = torch.float32 if shared.cmd_opts.no_half else torch.float16
+  devices.dtype_vae = torch.float32 if shared.cmd_opts.no_half or shared.cmd_opts.no_half_vae else torch.float16
+  devices.dtype_unet = model.model.diffusion_model.dtype
+  
+  if hasattr(shared.cmd_opts,"upcast_sampling"):
+      devices.unet_needs_upcast = shared.cmd_opts.upcast_sampling and devices.dtype == torch.float16 and devices.dtype_unet == torch.float16
+  else:
+      devices.unet_needs_upcast = devices.dtype == torch.float16 and devices.dtype_unet == torch.float16
+
+  model.first_stage_model.to(devices.dtype_vae)
+  sd_hijack.model_hijack.hijack(model)
+  
+  model.logvar = shared.sd_model.logvar.to(devices.device) 
+
+  if shared.cmd_opts.lowvram or shared.cmd_opts.medvram:
+      setup_for_low_vram_s(model, shared.cmd_opts.medvram)
+  else:
+      model.to(shared.device)
+
+  model.eval()
+
+  shared.sd_model = model
+  try:
+    sd_hijack.model_hijack.embedding_db.load_textual_inversion_embeddings(force_reload=True)
+  except:
+    pass
+  #shared.sd_model.sd_checkpoint_info.model_name = model_name
+  
+  def _setvae():
+      sd_vae.delete_base_vae()
+      sd_vae.clear_loaded_vae()
+      vae_file, vae_source = sd_vae.resolve_vae(model_a)
+      sd_vae.load_vae(shared.sd_model, vae_file, vae_source)
+
+  try:
+      _setvae()
+  except:
+      print("ERROR:setting VAE skipped")
+
+
+import torch
+from modules import devices
+
+module_in_gpu = None
+cpu = torch.device("cpu")
+
+
+def send_everything_to_cpu():
+    global module_in_gpu
+
+    if module_in_gpu is not None:
+        module_in_gpu.to(cpu)
+
+    module_in_gpu = None
+
+def setup_for_low_vram_s(sd_model, use_medvram):
+    parents = {}
+
+    def send_me_to_gpu(module, _):
+        """send this module to GPU; send whatever tracked module was previous in GPU to CPU;
+        we add this as forward_pre_hook to a lot of modules and this way all but one of them will
+        be in CPU
+        """
+        global module_in_gpu
+
+        module = parents.get(module, module)
+
+        if module_in_gpu == module:
+            return
+
+        if module_in_gpu is not None:
+            module_in_gpu.to(cpu)
+
+        module.to(devices.device)
+        module_in_gpu = module
+
+    # see below for register_forward_pre_hook;
+    # first_stage_model does not use forward(), it uses encode/decode, so register_forward_pre_hook is
+    # useless here, and we just replace those methods
+
+    first_stage_model = sd_model.first_stage_model
+    first_stage_model_encode = sd_model.first_stage_model.encode
+    first_stage_model_decode = sd_model.first_stage_model.decode
+
+    def first_stage_model_encode_wrap(x):
+        send_me_to_gpu(first_stage_model, None)
+        return first_stage_model_encode(x)
+
+    def first_stage_model_decode_wrap(z):
+        send_me_to_gpu(first_stage_model, None)
+        return first_stage_model_decode(z)
+
+    # for SD1, cond_stage_model is CLIP and its NN is in the tranformer frield, but for SD2, it's open clip, and it's in model field
+    if hasattr(sd_model.cond_stage_model, 'model'):
+        sd_model.cond_stage_model.transformer = sd_model.cond_stage_model.model
+
+    # remove four big modules, cond, first_stage, depth (if applicable), and unet from the model and then
+    # send the model to GPU. Then put modules back. the modules will be in CPU.
+    stored = sd_model.first_stage_model, getattr(sd_model, 'depth_model', None), sd_model.model
+    sd_model.first_stage_model, sd_model.depth_model, sd_model.model = None, None, None
+    sd_model.to(devices.device)
+    sd_model.first_stage_model, sd_model.depth_model, sd_model.model = stored
+
+    # register hooks for those the first three models
+    sd_model.first_stage_model.register_forward_pre_hook(send_me_to_gpu)
+    sd_model.first_stage_model.encode = first_stage_model_encode_wrap
+    sd_model.first_stage_model.decode = first_stage_model_decode_wrap
+    if sd_model.depth_model:
+        sd_model.depth_model.register_forward_pre_hook(send_me_to_gpu)
+
+    if hasattr(sd_model.cond_stage_model, 'model'):
+        sd_model.cond_stage_model.model = sd_model.cond_stage_model.transformer
+        del sd_model.cond_stage_model.transformer
+
+    if use_medvram:
+        sd_model.model.register_forward_pre_hook(send_me_to_gpu)
+    else:
+        diff_model = sd_model.model.diffusion_model
+
+        # the third remaining model is still too big for 4 GB, so we also do the same for its submodules
+        # so that only one of them is in GPU at a time
+        stored = diff_model.input_blocks, diff_model.middle_block, diff_model.output_blocks, diff_model.time_embed
+        diff_model.input_blocks, diff_model.middle_block, diff_model.output_blocks, diff_model.time_embed = None, None, None, None
+        sd_model.model.to(devices.device)
+        diff_model.input_blocks, diff_model.middle_block, diff_model.output_blocks, diff_model.time_embed = stored
+
+        # install hooks for bits of third model
+        diff_model.time_embed.register_forward_pre_hook(send_me_to_gpu)
+        for block in diff_model.input_blocks:
+            block.register_forward_pre_hook(send_me_to_gpu)
+        diff_model.middle_block.register_forward_pre_hook(send_me_to_gpu)
+        for block in diff_model.output_blocks:
+            block.register_forward_pre_hook(send_me_to_gpu)

microsoftexcel-supermerger/scripts/mergers/pluslora.py

@@ -0,0 +1,1298 @@
+import re
+from sklearn.linear_model import PassiveAggressiveClassifier
+import torch
+import math
+import os
+import gc
+import gradio as gr
+from torchmetrics import Precision
+import modules.shared as shared
+import gc
+from safetensors.torch import load_file, save_file
+from typing import List
+from tqdm import tqdm
+from modules import  sd_models,scripts
+from scripts.mergers.model_util import load_models_from_stable_diffusion_checkpoint,filenamecutter,savemodel
+from modules.ui import create_refresh_button
+
+def on_ui_tabs():
+    import lora
+    sml_path_root = scripts.basedir()
+    LWEIGHTSPRESETS="\
+    NONE:0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0\n\
+    ALL:1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n\
+    INS:1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0\n\
+    IND:1,0,0,0,1,1,1,0,0,0,0,0,0,0,0,0,0\n\
+    INALL:1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0\n\
+    MIDD:1,0,0,0,1,1,1,1,1,1,1,1,0,0,0,0,0\n\
+    OUTD:1,0,0,0,0,0,0,0,1,1,1,1,0,0,0,0,0\n\
+    OUTS:1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1\n\
+    OUTALL:1,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1\n\
+    ALL0.5:0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5"
+    sml_filepath = os.path.join(sml_path_root,"scripts", "lbwpresets.txt")
+    sml_lbwpresets=""
+    try:
+        with open(sml_filepath,encoding="utf-8") as f:
+            sml_lbwpresets = f.read()
+    except OSError as e:
+        sml_lbwpresets=LWEIGHTSPRESETS 
+
+    with gr.Blocks(analytics_enabled=False) :
+        sml_submit_result = gr.Textbox(label="Message")
+        with gr.Row().style(equal_height=False):
+            sml_cpmerge = gr.Button(elem_id="model_merger_merge", value="Merge to Checkpoint",variant='primary')
+            sml_makelora = gr.Button(elem_id="model_merger_merge", value="Make LoRA (alpha * A - beta * B)",variant='primary')
+            sml_model_a = gr.Dropdown(sd_models.checkpoint_tiles(),elem_id="model_converter_model_name",label="Checkpoint A",interactive=True)
+            create_refresh_button(sml_model_a, sd_models.list_models,lambda: {"choices": sd_models.checkpoint_tiles()},"refresh_checkpoint_Z")
+            sml_model_b = gr.Dropdown(sd_models.checkpoint_tiles(),elem_id="model_converter_model_name",label="Checkpoint B",interactive=True)
+            create_refresh_button(sml_model_b, sd_models.list_models,lambda: {"choices": sd_models.checkpoint_tiles()},"refresh_checkpoint_Z")
+        with gr.Row().style(equal_height=False):
+            sml_merge = gr.Button(elem_id="model_merger_merge", value="Merge LoRAs",variant='primary')
+            alpha = gr.Slider(label="alpha", minimum=-1.0, maximum=2, step=0.001, value=1)
+            beta = gr.Slider(label="beta", minimum=-1.0, maximum=2, step=0.001, value=1)
+        with gr.Row().style(equal_height=False):
+            sml_settings = gr.CheckboxGroup(["same to Strength", "overwrite"], label="settings")
+            precision = gr.Radio(label = "save precision",choices=["float","fp16","bf16"],value = "fp16",type="value")
+        with gr.Row().style(equal_height=False):
+          sml_dim = gr.Radio(label = "remake dimension",choices = ["no","auto",*[2**(x+2) for x in range(9)]],value = "no",type = "value") 
+          sml_filename = gr.Textbox(label="filename(option)",lines=1,visible =True,interactive  = True)  
+        sml_loranames = gr.Textbox(label='LoRAname1:ratio1:Blocks1,LoRAname2:ratio2:Blocks2,...(":blocks" is option, not necessary)',lines=1,value="",visible =True)
+        sml_dims = gr.CheckboxGroup(label = "limit dimension",choices=[],value = [],type="value",interactive=True,visible = False)
+        with gr.Row().style(equal_height=False):
+          sml_calcdim = gr.Button(elem_id="calcloras", value="calculate dimension of LoRAs(It may take a few minutes if there are many LoRAs)",variant='primary')
+          sml_update = gr.Button(elem_id="calcloras", value="update list",variant='primary')
+        sml_loras = gr.CheckboxGroup(label = "Lora",choices=[x[0] for x in lora.available_loras.items()],type="value",interactive=True,visible = True)
+        sml_loraratios = gr.TextArea(label="",value=sml_lbwpresets,visible =True,interactive  = True)  
+
+        sml_merge.click(
+            fn=lmerge,
+            inputs=[sml_loranames,sml_loraratios,sml_settings,sml_filename,sml_dim,precision],
+            outputs=[sml_submit_result]
+        )
+
+        sml_makelora.click(
+            fn=makelora,
+            inputs=[sml_model_a,sml_model_b,sml_dim,sml_filename,sml_settings,alpha,beta,precision],
+            outputs=[sml_submit_result]
+        )
+
+        sml_cpmerge.click(
+            fn=pluslora,
+            inputs=[sml_loranames,sml_loraratios,sml_settings,sml_filename,sml_model_a,precision],
+            outputs=[sml_submit_result]
+        )
+        llist ={}
+        dlist =[]
+        dn = []
+
+        def updateloras():
+          lora.list_available_loras()
+          for n in  lora.available_loras.items():
+            if n[0] not in llist:llist[n[0]] = ""
+          return gr.update(choices = [f"{x[0]}({x[1]})" for x in llist.items()])
+
+        sml_update.click(fn = updateloras,outputs = [sml_loras])
+
+        def calculatedim():
+          print("listing dimensions...")
+          for n in  tqdm(lora.available_loras.items()):
+              if n[0] in llist:
+                if llist[n[0]] !="": continue
+              c_lora = lora.available_loras.get(n[0], None) 
+              d,t = dimgetter(c_lora.filename)
+              if t == "LoCon" : d = f"{d}:{t}"
+              if d not in dlist:
+                if type(d) == int :dlist.append(d)
+                elif d not in dn: dn.append(d)
+              llist[n[0]] = d
+          dlist.sort()
+          return gr.update(choices = [f"{x[0]}({x[1]})" for x in llist.items()],value =[]),gr.update(visible =True,choices = [x for x in (dlist+dn)])
+
+        sml_calcdim.click(
+            fn=calculatedim,
+            inputs=[],
+            outputs=[sml_loras,sml_dims]
+        )
+
+        def dimselector(dims):
+          if dims ==[]:return gr.update(choices = [f"{x[0]}({x[1]})" for x in llist.items()])
+          rl=[]
+          for d in dims:
+            for i in llist.items():
+              if d == i[1]:rl.append(f"{i[0]}({i[1]})")
+          return gr.update(choices = [l for l in rl],value =[])
+
+        def llister(names):
+          if names ==[] : return ""
+          else:
+            for i,n in enumerate(names):
+              if "(" in n:names[i] = n[:n.rfind("(")]
+            return ":1.0,".join(names)+":1.0"
+        sml_loras.change(fn=llister,inputs=[sml_loras],outputs=[sml_loranames])     
+        sml_dims.change(fn=dimselector,inputs=[sml_dims],outputs=[sml_loras])  
+
+def makelora(model_a,model_b,dim,saveto,settings,alpha,beta,precision):
+    print("make LoRA start")
+    if model_a == "" or model_b =="":
+      return "ERROR: No model Selected"
+    gc.collect()
+
+    if saveto =="" : saveto = makeloraname(model_a,model_b)
+    if not ".safetensors" in saveto :saveto  += ".safetensors"
+    saveto = os.path.join(shared.cmd_opts.lora_dir,saveto)
+
+    dim = 128 if type(dim) != int else int(dim)
+    if os.path.isfile(saveto ) and not "overwrite" in settings:
+        _err_msg = f"Output file ({saveto}) existed and was not saved"
+        print(_err_msg)
+        return _err_msg
+
+    svd(fullpathfromname(model_a),fullpathfromname(model_b),False,dim,precision,saveto,alpha,beta)
+    return f"saved to {saveto}"
+
+def lmerge(loranames,loraratioss,settings,filename,dim,precision):
+    import lora
+    loras_on_disk = [lora.available_loras.get(name, None) for name in loranames]
+    if any([x is None for x in loras_on_disk]):
+        lora.list_available_loras()
+
+        loras_on_disk = [lora.available_loras.get(name, None) for name in loranames]
+
+    lnames = [loranames] if "," not in loranames else loranames.split(",")
+
+    for i, n in enumerate(lnames):
+        lnames[i] = n.split(":")
+
+    loraratios=loraratioss.splitlines()
+    ldict ={}
+
+    for i,l in enumerate(loraratios):
+        if ":" not in l or not (l.count(",") == 16 or l.count(",") == 25) : continue
+        ldict[l.split(":")[0]]=l.split(":")[1]
+
+    ln = []
+    lr = []
+    ld = []
+    lt = []
+    dmax = 1
+
+    for i,n in enumerate(lnames):
+        if len(n) ==3:
+            if n[2].strip() in ldict:
+                ratio = [float(r)*float(n[1]) for r in ldict[n[2]].split(",")]
+            else:ratio = [float(n[1])]*17
+        else:ratio = [float(n[1])]*17
+        c_lora = lora.available_loras.get(n[0], None) 
+        ln.append(c_lora.filename)
+        lr.append(ratio)
+        d,t = dimgetter(c_lora.filename)
+        lt.append(t)
+        ld.append(d)
+        if d != "LyCORIS":
+          if d > dmax : dmax = d
+
+    if filename =="":filename =loranames.replace(",","+").replace(":","_")
+    if not ".safetensors" in filename:filename += ".safetensors"
+    filename = os.path.join(shared.cmd_opts.lora_dir,filename)
+  
+    dim = int(dim) if dim != "no" and dim != "auto" else 0
+
+    if "LyCORIS" in ld or "LoCon" in lt:
+      if len(ld) !=1:
+        return "multiple merge of LyCORIS is not supported"
+      sd = lycomerge(ln[0],lr[0])
+    elif dim > 0:
+      print("change demension to ", dim)
+      sd = merge_lora_models_dim(ln, lr, dim,settings)
+    elif "auto" in settings and ld.count(ld[0]) != len(ld):
+      print("change demension to ",dmax)
+      sd = merge_lora_models_dim(ln, lr, dmax,settings)
+    else:
+      sd = merge_lora_models(ln, lr,settings)
+
+    if os.path.isfile(filename) and not "overwrite" in settings:
+        _err_msg = f"Output file ({filename}) existed and was not saved"
+        print(_err_msg)
+        return _err_msg
+
+    save_to_file(filename,sd,sd, str_to_dtype(precision))
+    return "saved : "+filename
+
+def pluslora(lnames,loraratios,settings,output,model,precision):
+    if model == []:
+      return "ERROR: No model Selected"
+    if lnames == "":
+      return "ERROR: No LoRA Selected"
+
+    print("plus LoRA start")
+    import lora
+    lnames = [lnames] if "," not in lnames else lnames.split(",")
+
+    for i, n in enumerate(lnames):
+        lnames[i] = n.split(":")
+
+    loraratios=loraratios.splitlines()
+    ldict ={}
+
+    for i,l in enumerate(loraratios):
+        if ":" not in l or not (l.count(",") == 16 or l.count(",") == 25) : continue
+        ldict[l.split(":")[0].strip()]=l.split(":")[1]
+
+    names=[]
+    filenames=[]
+    loratypes=[]
+    lweis=[]
+
+    for n in lnames:
+        if len(n) ==3:
+            if n[2].strip() in ldict:
+                ratio = [float(r)*float(n[1]) for r in ldict[n[2]].split(",")]
+            else:ratio = [float(n[1])]*17
+        else:ratio = [float(n[1])]*17
+        c_lora = lora.available_loras.get(n[0], None) 
+        names.append(n[0])
+        filenames.append(c_lora.filename)
+        _,t = dimgetter(c_lora.filename)
+        if "LyCORIS" in t: return "LyCORIS merge is not supported"
+        lweis.append(ratio)
+
+    modeln=filenamecutter(model,True)   
+    dname = modeln
+    for n in names:
+      dname = dname + "+"+n
+
+    checkpoint_info = sd_models.get_closet_checkpoint_match(model)
+    print(f"Loading {model}")
+    theta_0 = sd_models.read_state_dict(checkpoint_info.filename,"cpu")
+
+    keychanger = {}
+    for key in theta_0.keys():
+        if "model" in key:
+            skey = key.replace(".","_").replace("_weight","")
+            keychanger[skey.split("model_",1)[1]] = key
+
+    for name,filename, lwei in zip(names,filenames, lweis):
+      print(f"loading: {name}")
+      lora_sd = load_state_dict(filename, torch.float)
+
+      print(f"merging..." ,lwei)
+      for key in lora_sd.keys():
+        ratio = 1
+
+        fullkey = convert_diffusers_name_to_compvis(key)
+
+        for i,block in enumerate(LORABLOCKS):
+            if block in fullkey:
+                ratio = lwei[i]
+
+        msd_key, lora_key = fullkey.split(".", 1)
+
+        if "lora_down" in key:
+          up_key = key.replace("lora_down", "lora_up")
+          alpha_key = key[:key.index("lora_down")] + 'alpha'
+
+          # print(f"apply {key} to {module}")
+
+          down_weight = lora_sd[key].to(device="cpu")
+          up_weight = lora_sd[up_key].to(device="cpu")
+
+          dim = down_weight.size()[0]
+          alpha = lora_sd.get(alpha_key, dim)
+          scale = alpha / dim
+          # W <- W + U * D
+          weight = theta_0[keychanger[msd_key]].to(device="cpu")
+
+          if not len(down_weight.size()) == 4:
+            # linear
+            weight = weight  + ratio * (up_weight @ down_weight) * scale
+          else:
+            # conv2d
+            weight = weight  + ratio * (up_weight.squeeze(3).squeeze(2) @ down_weight.squeeze(3).squeeze(2)
+                                        ).unsqueeze(2).unsqueeze(3) * scale
+          theta_0[keychanger[msd_key]] = torch.nn.Parameter(weight)
+    #usemodelgen(theta_0,model)
+    settings.append(precision)
+    result = savemodel(theta_0,dname,output,settings,model)
+    del theta_0
+    gc.collect()
+    return result
+
+def save_to_file(file_name, model, state_dict, dtype):
+    if dtype is not None:
+        for key in list(state_dict.keys()):
+            if type(state_dict[key]) == torch.Tensor:
+                state_dict[key] = state_dict[key].to(dtype)
+
+    if os.path.splitext(file_name)[1] == '.safetensors':
+        save_file(model, file_name)
+    else:
+        torch.save(model, file_name)
+
+re_digits = re.compile(r"\d+")
+
+re_unet_down_blocks = re.compile(r"lora_unet_down_blocks_(\d+)_attentions_(\d+)_(.+)")
+re_unet_mid_blocks = re.compile(r"lora_unet_mid_block_attentions_(\d+)_(.+)")
+re_unet_up_blocks = re.compile(r"lora_unet_up_blocks_(\d+)_attentions_(\d+)_(.+)")
+
+re_unet_down_blocks_res = re.compile(r"lora_unet_down_blocks_(\d+)_resnets_(\d+)_(.+)")
+re_unet_mid_blocks_res = re.compile(r"lora_unet_mid_block_resnets_(\d+)_(.+)")
+re_unet_up_blocks_res = re.compile(r"lora_unet_up_blocks_(\d+)_resnets_(\d+)_(.+)")
+
+re_unet_downsample = re.compile(r"lora_unet_down_blocks_(\d+)_downsamplers_0_conv(.+)")
+re_unet_upsample = re.compile(r"lora_unet_up_blocks_(\d+)_upsamplers_0_conv(.+)")
+
+re_text_block = re.compile(r"lora_te_text_model_encoder_layers_(\d+)_(.+)")
+
+
+def convert_diffusers_name_to_compvis(key):
+    def match(match_list, regex):
+        r = re.match(regex, key)
+        if not r:
+            return False
+
+        match_list.clear()
+        match_list.extend([int(x) if re.match(re_digits, x) else x for x in r.groups()])
+        return True
+
+    m = []
+
+    if match(m, re_unet_down_blocks):
+        return f"diffusion_model_input_blocks_{1 + m[0] * 3 + m[1]}_1_{m[2]}"
+
+    if match(m, re_unet_mid_blocks):
+        return f"diffusion_model_middle_block_1_{m[1]}"
+
+    if match(m, re_unet_up_blocks):
+        return f"diffusion_model_output_blocks_{m[0] * 3 + m[1]}_1_{m[2]}"
+
+    if match(m, re_unet_down_blocks_res):
+        block = f"diffusion_model_input_blocks_{1 + m[0] * 3 + m[1]}_0_"
+        if m[2].startswith('conv1'):
+            return f"{block}in_layers_2{m[2][len('conv1'):]}"
+        elif m[2].startswith('conv2'):
+            return f"{block}out_layers_3{m[2][len('conv2'):]}"
+        elif m[2].startswith('time_emb_proj'):
+            return f"{block}emb_layers_1{m[2][len('time_emb_proj'):]}"
+        elif m[2].startswith('conv_shortcut'):
+            return f"{block}skip_connection{m[2][len('conv_shortcut'):]}"
+
+    if match(m, re_unet_mid_blocks_res):
+        block = f"diffusion_model_middle_block_{m[0]*2}_"
+        if m[1].startswith('conv1'):
+            return f"{block}in_layers_2{m[1][len('conv1'):]}"
+        elif m[1].startswith('conv2'):
+            return f"{block}out_layers_3{m[1][len('conv2'):]}"
+        elif m[1].startswith('time_emb_proj'):
+            return f"{block}emb_layers_1{m[1][len('time_emb_proj'):]}"
+        elif m[1].startswith('conv_shortcut'):
+            return f"{block}skip_connection{m[1][len('conv_shortcut'):]}"
+
+    if match(m, re_unet_up_blocks_res):
+        block = f"diffusion_model_output_blocks_{m[0] * 3 + m[1]}_0_"
+        if m[2].startswith('conv1'):
+            return f"{block}in_layers_2{m[2][len('conv1'):]}"
+        elif m[2].startswith('conv2'):
+            return f"{block}out_layers_3{m[2][len('conv2'):]}"
+        elif m[2].startswith('time_emb_proj'):
+            return f"{block}emb_layers_1{m[2][len('time_emb_proj'):]}"
+        elif m[2].startswith('conv_shortcut'):
+            return f"{block}skip_connection{m[2][len('conv_shortcut'):]}"
+
+    if match(m, re_unet_downsample):
+        return f"diffusion_model_input_blocks_{m[0]*3+3}_0_op{m[1]}"
+
+    if match(m, re_unet_upsample):
+        return f"diffusion_model_output_blocks_{m[0]*3 + 2}_{1+(m[0]!=0)}_conv{m[1]}"
+
+    if match(m, re_text_block):
+        return f"transformer_text_model_encoder_layers_{m[0]}_{m[1]}"
+
+    return key
+
+CLAMP_QUANTILE = 0.99
+MIN_DIFF = 1e-6
+
+def str_to_dtype(p):
+  if p == 'float':
+    return torch.float
+  if p == 'fp16':
+    return torch.float16
+  if p == 'bf16':
+    return torch.bfloat16
+  return None
+
+def svd(model_a,model_b,v2,dim,save_precision,save_to,alpha,beta):
+  save_dtype = str_to_dtype(save_precision)
+
+  if model_a == model_b:
+    text_encoder_t, _, unet_t = load_models_from_stable_diffusion_checkpoint(v2, model_a)
+    text_encoder_o, _, unet_o = text_encoder_t, _, unet_t
+  else:
+    print(f"loading SD model : {model_b}")
+    text_encoder_o, _, unet_o = load_models_from_stable_diffusion_checkpoint(v2, model_b)
+    
+    print(f"loading SD model : {model_a}")
+    text_encoder_t, _, unet_t = load_models_from_stable_diffusion_checkpoint(v2, model_a)
+
+  # create LoRA network to extract weights: Use dim (rank) as alpha
+  lora_network_o = create_network(1.0, dim, dim, None, text_encoder_o, unet_o)
+  lora_network_t = create_network(1.0, dim, dim, None, text_encoder_t, unet_t)
+  assert len(lora_network_o.text_encoder_loras) == len(
+      lora_network_t.text_encoder_loras), f"model version is different (SD1.x vs SD2.x) / それぞれのモデルのバージョンが違います（SD1.xベースとSD2.xベース） "
+  # get diffs
+  diffs = {}
+  text_encoder_different = False
+  for i, (lora_o, lora_t) in enumerate(zip(lora_network_o.text_encoder_loras, lora_network_t.text_encoder_loras)):
+    lora_name = lora_o.lora_name
+    module_o = lora_o.org_module
+    module_t = lora_t.org_module
+    diff = alpha*module_t.weight - beta*module_o.weight
+
+    # Text Encoder might be same
+    if torch.max(torch.abs(diff)) > MIN_DIFF:
+      text_encoder_different = True
+
+    diff = diff.float()
+    diffs[lora_name] = diff
+
+  if not text_encoder_different:
+    print("Text encoder is same. Extract U-Net only.")
+    lora_network_o.text_encoder_loras = []
+    diffs = {}
+
+  for i, (lora_o, lora_t) in enumerate(zip(lora_network_o.unet_loras, lora_network_t.unet_loras)):
+    lora_name = lora_o.lora_name
+    module_o = lora_o.org_module
+    module_t = lora_t.org_module
+    diff = alpha*module_t.weight - beta*module_o.weight
+    diff = diff.float()
+
+    diffs[lora_name] = diff
+
+  # make LoRA with svd
+  print("calculating by svd")
+  rank = dim
+  lora_weights = {}
+  with torch.no_grad():
+    for lora_name, mat in tqdm(list(diffs.items())):
+      conv2d = (len(mat.size()) == 4)
+      if conv2d:
+        mat = mat.squeeze()
+
+      U, S, Vh = torch.linalg.svd(mat)
+
+      U = U[:, :rank]
+      S = S[:rank]
+      U = U @ torch.diag(S)
+
+      Vh = Vh[:rank, :]
+
+      dist = torch.cat([U.flatten(), Vh.flatten()])
+      hi_val = torch.quantile(dist, CLAMP_QUANTILE)
+      low_val = -hi_val
+
+      U = U.clamp(low_val, hi_val)
+      Vh = Vh.clamp(low_val, hi_val)
+
+      lora_weights[lora_name] = (U, Vh)
+
+  # make state dict for LoRA
+  lora_network_o.apply_to(text_encoder_o, unet_o, text_encoder_different, True)   # to make state dict
+  lora_sd = lora_network_o.state_dict()
+  print(f"LoRA has {len(lora_sd)} weights.")
+
+  for key in list(lora_sd.keys()):
+    if "alpha" in key:
+      continue
+
+    lora_name = key.split('.')[0]
+    i = 0 if "lora_up" in key else 1
+
+    weights = lora_weights[lora_name][i]
+    # print(key, i, weights.size(), lora_sd[key].size())
+    if len(lora_sd[key].size()) == 4:
+      weights = weights.unsqueeze(2).unsqueeze(3)
+
+    assert weights.size() == lora_sd[key].size(), f"size unmatch: {key}"
+    lora_sd[key] = weights
+
+  # load state dict to LoRA and save it
+  info = lora_network_o.load_state_dict(lora_sd)
+  print(f"Loading extracted LoRA weights: {info}")
+
+  dir_name = os.path.dirname(save_to)
+  if dir_name and not os.path.exists(dir_name):
+    os.makedirs(dir_name, exist_ok=True)
+
+  # minimum metadata
+  metadata = {"ss_network_dim": str(dim), "ss_network_alpha": str(dim)}
+
+  lora_network_o.save_weights(save_to, save_dtype, metadata)
+  print(f"LoRA weights are saved to: {save_to}")
+  return save_to  
+
+def load_state_dict(file_name, dtype):
+  if os.path.splitext(file_name)[1] == '.safetensors':
+    sd = load_file(file_name)
+  else:
+    sd = torch.load(file_name, map_location='cpu')
+  for key in list(sd.keys()):
+    if type(sd[key]) == torch.Tensor:
+      sd[key] = sd[key].to(dtype)
+  return sd
+
+def dimgetter(filename):
+    lora_sd = load_state_dict(filename, torch.float)
+    alpha = None
+    dim = None
+    type = None
+
+    if "lora_unet_down_blocks_0_resnets_0_conv1.lora_down.weight" in lora_sd.keys():
+      type = "LoCon"
+
+    for key, value in lora_sd.items():
+  
+        if alpha is None and 'alpha' in key:
+            alpha = value
+        if dim is None and 'lora_down' in key and len(value.size()) == 2:
+            dim = value.size()[0]
+        if "hada_" in key:
+            dim,type = "LyCORIS","LyCORIS"
+        if alpha is not None and dim is not None:
+            break
+    if alpha is None:
+        alpha = dim
+    if type == None:type = "LoRA"
+    if dim :
+      return dim,type
+    else:
+      return "unknown","unknown"
+
+def blockfromkey(key):
+    fullkey = convert_diffusers_name_to_compvis(key)
+    for i,n in enumerate(LORABLOCKS):
+      if n in  fullkey: return i
+    return 0
+
+def merge_lora_models_dim(models, ratios, new_rank,sets):
+  merged_sd = {}
+  fugou = 1
+  for model, ratios in zip(models, ratios):
+    merge_dtype = torch.float
+    lora_sd = load_state_dict(model, merge_dtype)
+
+    # merge
+    print(f"merging {model}: {ratios}")
+    for key in tqdm(list(lora_sd.keys())):
+      if 'lora_down' not in key:
+        continue
+      lora_module_name = key[:key.rfind(".lora_down")]
+
+      down_weight = lora_sd[key]
+      network_dim = down_weight.size()[0]
+
+      up_weight = lora_sd[lora_module_name + '.lora_up.weight']
+      alpha = lora_sd.get(lora_module_name + '.alpha', network_dim)
+
+      in_dim = down_weight.size()[1]
+      out_dim = up_weight.size()[0]
+      conv2d = len(down_weight.size()) == 4
+     # print(lora_module_name, network_dim, alpha, in_dim, out_dim)
+
+      # make original weight if not exist
+      if lora_module_name not in merged_sd:
+        weight = torch.zeros((out_dim, in_dim, 1, 1) if conv2d else (out_dim, in_dim), dtype=merge_dtype)
+      else:
+        weight = merged_sd[lora_module_name]
+
+      ratio = ratios[blockfromkey(key)]
+      if "same to Strength" in sets:
+        ratio, fugou = (ratio**0.5,1) if ratio > 0 else (abs(ratio)**0.5,-1)
+      #print(lora_module_name, ratio)
+      # W <- W + U * D
+      scale = (alpha / network_dim)
+      if not conv2d:        # linear
+        weight = weight + ratio * (up_weight @ down_weight) * scale * fugou
+      else:
+        weight = weight + ratio * (up_weight.squeeze(3).squeeze(2) @ down_weight.squeeze(3).squeeze(2)
+                                   ).unsqueeze(2).unsqueeze(3) * scale * fugou
+
+      merged_sd[lora_module_name] = weight
+
+  # extract from merged weights
+  print("extract new lora...")
+  merged_lora_sd = {}
+  with torch.no_grad():
+    for lora_module_name, mat in tqdm(list(merged_sd.items())):
+      conv2d = (len(mat.size()) == 4)
+      if conv2d:
+        mat = mat.squeeze()
+
+      U, S, Vh = torch.linalg.svd(mat)
+
+      U = U[:, :new_rank]
+      S = S[:new_rank]
+      U = U @ torch.diag(S)
+
+      Vh = Vh[:new_rank, :]
+
+      dist = torch.cat([U.flatten(), Vh.flatten()])
+      hi_val = torch.quantile(dist, CLAMP_QUANTILE)
+      low_val = -hi_val
+
+      U = U.clamp(low_val, hi_val)
+      Vh = Vh.clamp(low_val, hi_val)
+
+      up_weight = U
+      down_weight = Vh
+
+      if conv2d:
+        up_weight = up_weight.unsqueeze(2).unsqueeze(3)
+        down_weight = down_weight.unsqueeze(2).unsqueeze(3)
+
+      merged_lora_sd[lora_module_name + '.lora_up.weight'] = up_weight.to("cpu").contiguous()
+      merged_lora_sd[lora_module_name + '.lora_down.weight'] = down_weight.to("cpu").contiguous()
+      merged_lora_sd[lora_module_name + '.alpha'] = torch.tensor(new_rank)
+
+  return merged_lora_sd
+
+def merge_lora_models(models, ratios,sets):
+  base_alphas = {}                          # alpha for merged model
+  base_dims = {}
+  merge_dtype = torch.float
+  merged_sd = {}
+  fugou = 1
+  for model, ratios in zip(models, ratios):
+    print(f"merging {model}: {ratios}")
+    lora_sd = load_state_dict(model, merge_dtype)
+
+    # get alpha and dim
+    alphas = {}                             # alpha for current model
+    dims = {}                               # dims for current model
+    for key in lora_sd.keys():
+      if 'alpha' in key:
+        lora_module_name = key[:key.rfind(".alpha")]
+        alpha = float(lora_sd[key].detach().numpy())
+        alphas[lora_module_name] = alpha
+        if lora_module_name not in base_alphas:
+          base_alphas[lora_module_name] = alpha
+      elif "lora_down" in key:
+        lora_module_name = key[:key.rfind(".lora_down")]
+        dim = lora_sd[key].size()[0]
+        dims[lora_module_name] = dim
+        if lora_module_name not in base_dims:
+          base_dims[lora_module_name] = dim
+
+    for lora_module_name in dims.keys():
+      if lora_module_name not in alphas:
+        alpha = dims[lora_module_name]
+        alphas[lora_module_name] = alpha
+        if lora_module_name not in base_alphas:
+          base_alphas[lora_module_name] = alpha
+    
+    print(f"dim: {list(set(dims.values()))}, alpha: {list(set(alphas.values()))}")
+
+    # merge
+    print(f"merging...")
+    for key in lora_sd.keys():
+      if 'alpha' in key:
+        continue
+      if "lora_down" in key: dwon = True
+      lora_module_name = key[:key.rfind(".lora_")]
+
+      base_alpha = base_alphas[lora_module_name]
+      alpha = alphas[lora_module_name]
+
+      ratio = ratios[blockfromkey(key)]
+      if "same to Strength" in sets:
+        ratio, fugou = (ratio**0.5,1) if ratio > 0 else (abs(ratio)**0.5,-1)
+
+      if "lora_down" in key:
+        ratio = ratio * fugou
+
+      scale = math.sqrt(alpha / base_alpha) * ratio
+
+      if key in merged_sd:
+        assert merged_sd[key].size() == lora_sd[key].size(
+        ), f"weights shape mismatch merging v1 and v2, different dims? / 重みのサイズが合いません。v1とv2、または次元数の異なるモデルはマージできません"
+        merged_sd[key] = merged_sd[key] + lora_sd[key] * scale
+      else:
+        merged_sd[key] = lora_sd[key] * scale
+  
+  # set alpha to sd
+  for lora_module_name, alpha in base_alphas.items():
+    key = lora_module_name + ".alpha"
+    merged_sd[key] = torch.tensor(alpha)
+
+  print("merged model")
+  print(f"dim: {list(set(base_dims.values()))}, alpha: {list(set(base_alphas.values()))}")
+
+  return merged_sd
+
+def fullpathfromname(name):
+    if hash == "" or hash ==[]: return ""
+    checkpoint_info = sd_models.get_closet_checkpoint_match(name)
+    return checkpoint_info.filename
+
+def makeloraname(model_a,model_b):
+    model_a=filenamecutter(model_a)
+    model_b=filenamecutter(model_b)
+    return "lora_"+model_a+"-"+model_b
+
+def lycomerge(filename,ratios):
+    sd = load_state_dict(filename, torch.float)
+
+    if len(ratios) == 17:
+      r0 = 1
+      ratios = [ratios[0]] + [r0] + ratios[1:3]+ [r0] + ratios[3:5]+[r0] + ratios[5:7]+[r0,r0,r0] + [ratios[7]] + [r0,r0,r0] + ratios[8:]
+
+    print("LyCORIS: " , ratios)
+
+    keys_failed_to_match = []
+
+    for lkey, weight in sd.items():
+        ratio = 1
+        picked = False
+        if 'alpha' in lkey:
+          continue
+        
+        fullkey = convert_diffusers_name_to_compvis(lkey)
+        key, lora_key = fullkey.split(".", 1)
+
+        for i,block in enumerate(LYCOBLOCKS):
+            if block in key:
+                ratio = ratios[i]
+                picked = True
+        if not picked: keys_failed_to_match.append(key)
+
+        sd[lkey] = weight * math.sqrt(abs(float(ratio)))
+
+        if "down" in lkey and ratio < 0:
+          sd[key] = sd[key] * -1
+        
+    if len(keys_failed_to_match) > 0:
+      print(keys_failed_to_match)
+  
+    return sd 
+
+LORABLOCKS=["encoder",
+"diffusion_model_input_blocks_1_",
+"diffusion_model_input_blocks_2_",
+"diffusion_model_input_blocks_4_",
+"diffusion_model_input_blocks_5_",
+"diffusion_model_input_blocks_7_",
+"diffusion_model_input_blocks_8_",
+"diffusion_model_middle_block_",
+"diffusion_model_output_blocks_3_",
+"diffusion_model_output_blocks_4_",
+"diffusion_model_output_blocks_5_",
+"diffusion_model_output_blocks_6_",
+"diffusion_model_output_blocks_7_",
+"diffusion_model_output_blocks_8_",
+"diffusion_model_output_blocks_9_",
+"diffusion_model_output_blocks_10_",
+"diffusion_model_output_blocks_11_"]
+
+LYCOBLOCKS=["encoder",
+"diffusion_model_input_blocks_0_",
+"diffusion_model_input_blocks_1_",
+"diffusion_model_input_blocks_2_",
+"diffusion_model_input_blocks_3_",
+"diffusion_model_input_blocks_4_",
+"diffusion_model_input_blocks_5_",
+"diffusion_model_input_blocks_6_",
+"diffusion_model_input_blocks_7_",
+"diffusion_model_input_blocks_8_",
+"diffusion_model_input_blocks_9_",
+"diffusion_model_input_blocks_10_",
+"diffusion_model_input_blocks_11_",
+"diffusion_model_middle_block_",
+"diffusion_model_output_blocks_0_",
+"diffusion_model_output_blocks_1_",
+"diffusion_model_output_blocks_2_",
+"diffusion_model_output_blocks_3_",
+"diffusion_model_output_blocks_4_",
+"diffusion_model_output_blocks_5_",
+"diffusion_model_output_blocks_6_",
+"diffusion_model_output_blocks_7_",
+"diffusion_model_output_blocks_8_",
+"diffusion_model_output_blocks_9_",
+"diffusion_model_output_blocks_10_",
+"diffusion_model_output_blocks_11_"]
+
+class LoRAModule(torch.nn.Module):
+    """
+    replaces forward method of the original Linear, instead of replacing the original Linear module.
+    """
+
+    def __init__(self, lora_name, org_module: torch.nn.Module, multiplier=1.0, lora_dim=4, alpha=1):
+        """if alpha == 0 or None, alpha is rank (no scaling)."""
+        super().__init__()
+        self.lora_name = lora_name
+
+        if org_module.__class__.__name__ == "Conv2d":
+            in_dim = org_module.in_channels
+            out_dim = org_module.out_channels
+        else:
+            in_dim = org_module.in_features
+            out_dim = org_module.out_features
+
+        # if limit_rank:
+        #   self.lora_dim = min(lora_dim, in_dim, out_dim)
+        #   if self.lora_dim != lora_dim:
+        #     print(f"{lora_name} dim (rank) is changed to: {self.lora_dim}")
+        # else:
+        self.lora_dim = lora_dim
+
+        if org_module.__class__.__name__ == "Conv2d":
+            kernel_size = org_module.kernel_size
+            stride = org_module.stride
+            padding = org_module.padding
+            self.lora_down = torch.nn.Conv2d(in_dim, self.lora_dim, kernel_size, stride, padding, bias=False)
+            self.lora_up = torch.nn.Conv2d(self.lora_dim, out_dim, (1, 1), (1, 1), bias=False)
+        else:
+            self.lora_down = torch.nn.Linear(in_dim, self.lora_dim, bias=False)
+            self.lora_up = torch.nn.Linear(self.lora_dim, out_dim, bias=False)
+
+        if type(alpha) == torch.Tensor:
+            alpha = alpha.detach().float().numpy()  # without casting, bf16 causes error
+        alpha = self.lora_dim if alpha is None or alpha == 0 else alpha
+        self.scale = alpha / self.lora_dim
+        self.register_buffer("alpha", torch.tensor(alpha))  # 定数として扱える
+
+        # same as microsoft's
+        torch.nn.init.kaiming_uniform_(self.lora_down.weight, a=math.sqrt(5))
+        torch.nn.init.zeros_(self.lora_up.weight)
+
+        self.multiplier = multiplier
+        self.org_module = org_module  # remove in applying
+        self.region = None
+        self.region_mask = None
+
+    def apply_to(self):
+        self.org_forward = self.org_module.forward
+        self.org_module.forward = self.forward
+        del self.org_module
+
+    def merge_to(self, sd, dtype, device):
+        # get up/down weight
+        up_weight = sd["lora_up.weight"].to(torch.float).to(device)
+        down_weight = sd["lora_down.weight"].to(torch.float).to(device)
+
+        # extract weight from org_module
+        org_sd = self.org_module.state_dict()
+        weight = org_sd["weight"].to(torch.float)
+
+        # merge weight
+        if len(weight.size()) == 2:
+            # linear
+            weight = weight + self.multiplier * (up_weight @ down_weight) * self.scale
+        elif down_weight.size()[2:4] == (1, 1):
+            # conv2d 1x1
+            weight = (
+                weight
+                + self.multiplier
+                * (up_weight.squeeze(3).squeeze(2) @ down_weight.squeeze(3).squeeze(2)).unsqueeze(2).unsqueeze(3)
+                * self.scale
+            )
+        else:
+            # conv2d 3x3
+            conved = torch.nn.functional.conv2d(down_weight.permute(1, 0, 2, 3), up_weight).permute(1, 0, 2, 3)
+            # print(conved.size(), weight.size(), module.stride, module.padding)
+            weight = weight + self.multiplier * conved * self.scale
+
+        # set weight to org_module
+        org_sd["weight"] = weight.to(dtype)
+        self.org_module.load_state_dict(org_sd)
+
+    def set_region(self, region):
+        self.region = region
+        self.region_mask = None
+
+    def forward(self, x):
+        if self.region is None:
+            return self.org_forward(x) + self.lora_up(self.lora_down(x)) * self.multiplier * self.scale
+
+        # regional LoRA   FIXME same as additional-network extension
+        if x.size()[1] % 77 == 0:
+            # print(f"LoRA for context: {self.lora_name}")
+            self.region = None
+            return self.org_forward(x) + self.lora_up(self.lora_down(x)) * self.multiplier * self.scale
+
+        # calculate region mask first time
+        if self.region_mask is None:
+            if len(x.size()) == 4:
+                h, w = x.size()[2:4]
+            else:
+                seq_len = x.size()[1]
+                ratio = math.sqrt((self.region.size()[0] * self.region.size()[1]) / seq_len)
+                h = int(self.region.size()[0] / ratio + 0.5)
+                w = seq_len // h
+
+            r = self.region.to(x.device)
+            if r.dtype == torch.bfloat16:
+                r = r.to(torch.float)
+            r = r.unsqueeze(0).unsqueeze(1)
+            # print(self.lora_name, self.region.size(), x.size(), r.size(), h, w)
+            r = torch.nn.functional.interpolate(r, (h, w), mode="bilinear")
+            r = r.to(x.dtype)
+
+            if len(x.size()) == 3:
+                r = torch.reshape(r, (1, x.size()[1], -1))
+
+            self.region_mask = r
+
+        return self.org_forward(x) + self.lora_up(self.lora_down(x)) * self.multiplier * self.scale * self.region_mask
+
+def create_network(multiplier, network_dim, network_alpha, vae, text_encoder, unet, **kwargs):
+    if network_dim is None:
+        network_dim = 4  # default
+
+    # extract dim/alpha for conv2d, and block dim
+    conv_dim = kwargs.get("conv_dim", None)
+    conv_alpha = kwargs.get("conv_alpha", None)
+    if conv_dim is not None:
+        conv_dim = int(conv_dim)
+        if conv_alpha is None:
+            conv_alpha = 1.0
+        else:
+            conv_alpha = float(conv_alpha)
+
+    """
+    block_dims = kwargs.get("block_dims")
+    block_alphas = None
+    if block_dims is not None:
+    block_dims = [int(d) for d in block_dims.split(',')]
+    assert len(block_dims) == NUM_BLOCKS, f"Number of block dimensions is not same to {NUM_BLOCKS}"
+    block_alphas = kwargs.get("block_alphas")
+    if block_alphas is None:
+        block_alphas = [1] * len(block_dims)
+    else:
+        block_alphas = [int(a) for a in block_alphas(',')]
+    assert len(block_alphas) == NUM_BLOCKS, f"Number of block alphas is not same to {NUM_BLOCKS}"
+    conv_block_dims = kwargs.get("conv_block_dims")
+    conv_block_alphas = None
+    if conv_block_dims is not None:
+    conv_block_dims = [int(d) for d in conv_block_dims.split(',')]
+    assert len(conv_block_dims) == NUM_BLOCKS, f"Number of block dimensions is not same to {NUM_BLOCKS}"
+    conv_block_alphas = kwargs.get("conv_block_alphas")
+    if conv_block_alphas is None:
+        conv_block_alphas = [1] * len(conv_block_dims)
+    else:
+        conv_block_alphas = [int(a) for a in conv_block_alphas(',')]
+    assert len(conv_block_alphas) == NUM_BLOCKS, f"Number of block alphas is not same to {NUM_BLOCKS}"
+  """
+
+    network = LoRANetwork(
+        text_encoder,
+        unet,
+        multiplier=multiplier,
+        lora_dim=network_dim,
+        alpha=network_alpha,
+        conv_lora_dim=conv_dim,
+        conv_alpha=conv_alpha,
+    )
+    return network
+
+
+
+class LoRANetwork(torch.nn.Module):
+    # is it possible to apply conv_in and conv_out?
+    UNET_TARGET_REPLACE_MODULE = ["Transformer2DModel", "Attention"]
+    UNET_TARGET_REPLACE_MODULE_CONV2D_3X3 = ["ResnetBlock2D", "Downsample2D", "Upsample2D"]
+    TEXT_ENCODER_TARGET_REPLACE_MODULE = ["CLIPAttention", "CLIPMLP"]
+    LORA_PREFIX_UNET = "lora_unet"
+    LORA_PREFIX_TEXT_ENCODER = "lora_te"
+
+    def __init__(
+        self,
+        text_encoder,
+        unet,
+        multiplier=1.0,
+        lora_dim=4,
+        alpha=1,
+        conv_lora_dim=None,
+        conv_alpha=None,
+        modules_dim=None,
+        modules_alpha=None,
+    ) -> None:
+        super().__init__()
+        self.multiplier = multiplier
+
+        self.lora_dim = lora_dim
+        self.alpha = alpha
+        self.conv_lora_dim = conv_lora_dim
+        self.conv_alpha = conv_alpha
+
+        if modules_dim is not None:
+            print(f"create LoRA network from weights")
+        else:
+            print(f"create LoRA network. base dim (rank): {lora_dim}, alpha: {alpha}")
+
+        self.apply_to_conv2d_3x3 = self.conv_lora_dim is not None
+        if self.apply_to_conv2d_3x3:
+            if self.conv_alpha is None:
+                self.conv_alpha = self.alpha
+            print(f"apply LoRA to Conv2d with kernel size (3,3). dim (rank): {self.conv_lora_dim}, alpha: {self.conv_alpha}")
+
+        # create module instances
+        def create_modules(prefix, root_module: torch.nn.Module, target_replace_modules) -> List[LoRAModule]:
+            loras = []
+            for name, module in root_module.named_modules():
+                if module.__class__.__name__ in target_replace_modules:
+                    # TODO get block index here
+                    for child_name, child_module in module.named_modules():
+                        is_linear = child_module.__class__.__name__ == "Linear"
+                        is_conv2d = child_module.__class__.__name__ == "Conv2d"
+                        is_conv2d_1x1 = is_conv2d and child_module.kernel_size == (1, 1)
+                        if is_linear or is_conv2d:
+                            lora_name = prefix + "." + name + "." + child_name
+                            lora_name = lora_name.replace(".", "_")
+
+                            if modules_dim is not None:
+                                if lora_name not in modules_dim:
+                                    continue  # no LoRA module in this weights file
+                                dim = modules_dim[lora_name]
+                                alpha = modules_alpha[lora_name]
+                            else:
+                                if is_linear or is_conv2d_1x1:
+                                    dim = self.lora_dim
+                                    alpha = self.alpha
+                                elif self.apply_to_conv2d_3x3:
+                                    dim = self.conv_lora_dim
+                                    alpha = self.conv_alpha
+                                else:
+                                    continue
+
+                            lora = LoRAModule(lora_name, child_module, self.multiplier, dim, alpha)
+                            loras.append(lora)
+            return loras
+
+        self.text_encoder_loras = create_modules(
+            LoRANetwork.LORA_PREFIX_TEXT_ENCODER, text_encoder, LoRANetwork.TEXT_ENCODER_TARGET_REPLACE_MODULE
+        )
+        print(f"create LoRA for Text Encoder: {len(self.text_encoder_loras)} modules.")
+
+        # extend U-Net target modules if conv2d 3x3 is enabled, or load from weights
+        target_modules = LoRANetwork.UNET_TARGET_REPLACE_MODULE
+        if modules_dim is not None or self.conv_lora_dim is not None:
+            target_modules += LoRANetwork.UNET_TARGET_REPLACE_MODULE_CONV2D_3X3
+
+        self.unet_loras = create_modules(LoRANetwork.LORA_PREFIX_UNET, unet, target_modules)
+        print(f"create LoRA for U-Net: {len(self.unet_loras)} modules.")
+
+        self.weights_sd = None
+
+        # assertion
+        names = set()
+        for lora in self.text_encoder_loras + self.unet_loras:
+            assert lora.lora_name not in names, f"duplicated lora name: {lora.lora_name}"
+            names.add(lora.lora_name)
+
+    def set_multiplier(self, multiplier):
+        self.multiplier = multiplier
+        for lora in self.text_encoder_loras + self.unet_loras:
+            lora.multiplier = self.multiplier
+
+    def load_weights(self, file):
+        if os.path.splitext(file)[1] == ".safetensors":
+            from safetensors.torch import load_file, safe_open
+
+            self.weights_sd = load_file(file)
+        else:
+            self.weights_sd = torch.load(file, map_location="cpu")
+
+    def apply_to(self, text_encoder, unet, apply_text_encoder=None, apply_unet=None):
+        if self.weights_sd:
+            weights_has_text_encoder = weights_has_unet = False
+            for key in self.weights_sd.keys():
+                if key.startswith(LoRANetwork.LORA_PREFIX_TEXT_ENCODER):
+                    weights_has_text_encoder = True
+                elif key.startswith(LoRANetwork.LORA_PREFIX_UNET):
+                    weights_has_unet = True
+
+            if apply_text_encoder is None:
+                apply_text_encoder = weights_has_text_encoder
+            else:
+                assert (
+                    apply_text_encoder == weights_has_text_encoder
+                ), f"text encoder weights: {weights_has_text_encoder} but text encoder flag: {apply_text_encoder} / 重みとText Encoderのフラグが矛盾しています"
+
+            if apply_unet is None:
+                apply_unet = weights_has_unet
+            else:
+                assert (
+                    apply_unet == weights_has_unet
+                ), f"u-net weights: {weights_has_unet} but u-net flag: {apply_unet} / 重みとU-Netのフラグが矛盾しています"
+        else:
+            assert apply_text_encoder is not None and apply_unet is not None, f"internal error: flag not set"
+
+        if apply_text_encoder:
+            print("enable LoRA for text encoder")
+        else:
+            self.text_encoder_loras = []
+
+        if apply_unet:
+            print("enable LoRA for U-Net")
+        else:
+            self.unet_loras = []
+
+        for lora in self.text_encoder_loras + self.unet_loras:
+            lora.apply_to()
+            self.add_module(lora.lora_name, lora)
+
+        if self.weights_sd:
+            # if some weights are not in state dict, it is ok because initial LoRA does nothing (lora_up is initialized by zeros)
+            info = self.load_state_dict(self.weights_sd, False)
+            print(f"weights are loaded: {info}")
+
+    # TODO refactor to common function with apply_to
+    def merge_to(self, text_encoder, unet, dtype, device):
+        assert self.weights_sd is not None, "weights are not loaded"
+
+        apply_text_encoder = apply_unet = False
+        for key in self.weights_sd.keys():
+            if key.startswith(LoRANetwork.LORA_PREFIX_TEXT_ENCODER):
+                apply_text_encoder = True
+            elif key.startswith(LoRANetwork.LORA_PREFIX_UNET):
+                apply_unet = True
+
+        if apply_text_encoder:
+            print("enable LoRA for text encoder")
+        else:
+            self.text_encoder_loras = []
+
+        if apply_unet:
+            print("enable LoRA for U-Net")
+        else:
+            self.unet_loras = []
+
+        for lora in self.text_encoder_loras + self.unet_loras:
+            sd_for_lora = {}
+            for key in self.weights_sd.keys():
+                if key.startswith(lora.lora_name):
+                    sd_for_lora[key[len(lora.lora_name) + 1 :]] = self.weights_sd[key]
+            lora.merge_to(sd_for_lora, dtype, device)
+        print(f"weights are merged")
+
+    def enable_gradient_checkpointing(self):
+        # not supported
+        pass
+
+    def prepare_optimizer_params(self, text_encoder_lr, unet_lr):
+        def enumerate_params(loras):
+            params = []
+            for lora in loras:
+                params.extend(lora.parameters())
+            return params
+
+        self.requires_grad_(True)
+        all_params = []
+
+        if self.text_encoder_loras:
+            param_data = {"params": enumerate_params(self.text_encoder_loras)}
+            if text_encoder_lr is not None:
+                param_data["lr"] = text_encoder_lr
+            all_params.append(param_data)
+
+        if self.unet_loras:
+            param_data = {"params": enumerate_params(self.unet_loras)}
+            if unet_lr is not None:
+                param_data["lr"] = unet_lr
+            all_params.append(param_data)
+
+        return all_params
+
+    def prepare_grad_etc(self, text_encoder, unet):
+        self.requires_grad_(True)
+
+    def on_epoch_start(self, text_encoder, unet):
+        self.train()
+
+    def get_trainable_params(self):
+        return self.parameters()
+
+    def save_weights(self, file, dtype, metadata):
+        if metadata is not None and len(metadata) == 0:
+            metadata = None
+
+        state_dict = self.state_dict()
+
+        if dtype is not None:
+            for key in list(state_dict.keys()):
+                v = state_dict[key]
+                v = v.detach().clone().to("cpu").to(dtype)
+                state_dict[key] = v
+
+        if os.path.splitext(file)[1] == ".safetensors":
+            from safetensors.torch import save_file
+
+            # Precalculate model hashes to save time on indexing
+            if metadata is None:
+                metadata = {}
+            model_hash, legacy_hash = precalculate_safetensors_hashes(state_dict, metadata)
+            metadata["sshs_model_hash"] = model_hash
+            metadata["sshs_legacy_hash"] = legacy_hash
+
+            save_file(state_dict, file, metadata)
+        else:
+            torch.save(state_dict, file)
+
+    @staticmethod
+    def set_regions(networks, image):
+        image = image.astype(np.float32) / 255.0
+        for i, network in enumerate(networks[:3]):
+            # NOTE: consider averaging overwrapping area
+            region = image[:, :, i]
+            if region.max() == 0:
+                continue
+            region = torch.tensor(region)
+            network.set_region(region)
+
+    def set_region(self, region):
+        for lora in self.unet_loras:
+            lora.set_region(region)
+
+from io import BytesIO
+import safetensors.torch
+import hashlib
+
+def precalculate_safetensors_hashes(tensors, metadata):
+    """Precalculate the model hashes needed by sd-webui-additional-networks to
+    save time on indexing the model later."""
+
+    # Because writing user metadata to the file can change the result of
+    # sd_models.model_hash(), only retain the training metadata for purposes of
+    # calculating the hash, as they are meant to be immutable
+    metadata = {k: v for k, v in metadata.items() if k.startswith("ss_")}
+
+    bytes = safetensors.torch.save(tensors, metadata)
+    b = BytesIO(bytes)
+
+    model_hash = addnet_hash_safetensors(b)
+    legacy_hash = addnet_hash_legacy(b)
+    return model_hash, legacy_hash
+
+def addnet_hash_safetensors(b):
+    """New model hash used by sd-webui-additional-networks for .safetensors format files"""
+    hash_sha256 = hashlib.sha256()
+    blksize = 1024 * 1024
+
+    b.seek(0)
+    header = b.read(8)
+    n = int.from_bytes(header, "little")
+
+    offset = n + 8
+    b.seek(offset)
+    for chunk in iter(lambda: b.read(blksize), b""):
+        hash_sha256.update(chunk)
+
+    return hash_sha256.hexdigest()
+
+def addnet_hash_legacy(b):
+    """Old model hash used by sd-webui-additional-networks for .safetensors format files"""
+    m = hashlib.sha256()
+
+    b.seek(0x100000)
+    m.update(b.read(0x10000))
+    return m.hexdigest()[0:8]

microsoftexcel-supermerger/scripts/mergers/xyplot.py

@@ -0,0 +1,513 @@
+import random
+import cv2
+import numpy as np
+import os
+import copy
+import csv
+from PIL import Image
+from modules import images
+from modules.shared import opts
+from scripts.mergers.mergers import TYPES,smerge,simggen,filenamecutter,draw_origin,wpreseter
+from scripts.mergers.model_util import usemodelgen
+
+hear = True
+hearm = False
+
+state_mergen = False
+
+numadepth = []
+
+def freezetime():
+    global state_mergen
+    state_mergen = True
+
+def numanager(normalstart,xtype,xmen,ytype,ymen,esettings,
+                    weights_a,weights_b,model_a,model_b,model_c,alpha,beta,mode,calcmode,useblocks,custom_name,save_sets,id_sets,wpresets,deep,tensor,
+                    prompt,nprompt,steps,sampler,cfg,seed,w,h,
+                    hireson,hrupscaler,hr2ndsteps,denoise_str,hr_scale,batch_size):
+    global numadepth
+    grids = []
+    sep = "|"
+
+    if sep  in xmen:
+        xmens = xmen.split(sep)
+        xmen = xmens[0]
+        if seed =="-1": seed = str(random.randrange(4294967294))
+        for men in xmens[1:]:
+            numaker(xtype,men,ytype,ymen,esettings,
+                        weights_a,weights_b,model_a,model_b,model_c,alpha,beta,mode,calcmode,useblocks,custom_name,save_sets,id_sets,wpresets,deep,tensor,
+                        prompt,nprompt,steps,sampler,cfg,seed,w,h,
+                        hireson,hrupscaler,hr2ndsteps,denoise_str,hr_scale,batch_size)
+    elif sep  in ymen:
+        ymens = ymen.split(sep)
+        ymen = ymens[0]
+        if seed =="-1": seed = str(random.randrange(4294967294))
+        for men in ymens[1:]:
+            numaker(xtype,xmen,ytype,men,esettings,
+                        weights_a,weights_b,model_a,model_b,model_c,alpha,beta,mode,calcmode,useblocks,custom_name,save_sets,id_sets,wpresets,deep,tensor,
+                        prompt,nprompt,steps,sampler,cfg,seed,w,h,
+                        hireson,hrupscaler,hr2ndsteps,denoise_str,hr_scale,batch_size)
+
+    if normalstart:
+        result,currentmodel,xyimage,a,b,c= sgenxyplot(xtype,xmen,ytype,ymen,esettings,
+                                                                             weights_a,weights_b,model_a,model_b,model_c,alpha,beta,mode,calcmode,
+                                                                             useblocks,custom_name,save_sets,id_sets,wpresets,deep,tensor,
+                                                                             prompt,nprompt,steps,sampler,cfg,seed,w,h,
+                                                                             hireson,hrupscaler,hr2ndsteps,denoise_str,hr_scale,batch_size)
+        if xyimage is not None:grids =[xyimage[0]]
+        else:print(result)
+    else:
+        if numadepth ==[]:
+            return "no reservation",*[None]*5
+        result=currentmodel=xyimage=a=b=c = None
+
+    while True:
+        for i,row in enumerate(numadepth):
+            if row[1] =="waiting":  
+                numadepth[i][1] = "Operating"
+                try:
+                    result,currentmodel,xyimage,a,b,c = sgenxyplot(*row[2:])
+                except Exception as e:
+                    print(e)
+                    numadepth[i][1] = "Error"
+                else:
+                    if xyimage is not None:
+                        grids.append(xyimage[0])
+                        numadepth[i][1] = "Finished"
+                    else:
+                        print(result)
+                        numadepth[i][1] = "Error"
+        wcounter = 0
+        for row in numadepth:
+            if row[1] != "waiting":
+                wcounter += 1
+        if wcounter == len(numadepth):
+            break
+
+    return result,currentmodel,grids,a,b,c
+
+def numaker(xtype,xmen,ytype,ymen,esettings,
+#msettings=[weights_a,weights_b,model_a,model_b,model_c,alpha,beta,mode,calcmode,useblocks,custom_name,save_sets,id_sets,wpresets]       
+                    weights_a,weights_b,model_a,model_b,model_c,alpha,beta,mode,calcmode,
+                    useblocks,custom_name,save_sets,id_sets,wpresets,deep,tensor,
+                    prompt,nprompt,steps,sampler,cfg,seed,w,h,
+                    hireson,hrupscaler,hr2ndsteps,denoise_str,hr_scale,batch_size):
+    global numadepth
+    numadepth.append([len(numadepth)+1,"waiting",xtype,xmen,ytype,ymen,esettings,
+                    weights_a,weights_b,model_a,model_b,model_c,alpha,beta,mode,calcmode,
+                    useblocks,custom_name,save_sets,id_sets,wpresets,deep,tensor,
+                    prompt,nprompt,steps,sampler,cfg,seed,w,h,
+                    hireson,hrupscaler,hr2ndsteps,denoise_str,hr_scale,batch_size])
+    return numalistmaker(copy.deepcopy(numadepth))
+
+def nulister(redel):
+    global numadepth
+    if redel == False:
+        return numalistmaker(copy.deepcopy(numadepth))
+    if redel ==-1:
+        numadepth = []
+    else:
+        try:del numadepth[int(redel-1)]
+        except Exception as e:print(e)
+    return numalistmaker(copy.deepcopy(numadepth))
+
+def numalistmaker(numa):
+    if numa ==[]: return [["no data","",""],]
+    for i,r in enumerate(numa):
+        r[2] =  TYPES[int(r[2])]
+        r[4] =  TYPES[int(r[4])]
+        numa[i] = r[0:6]+r[8:11]+r[12:16]+r[6:8]
+    return numa
+
+def caster(news,hear):
+    if hear: print(news)
+
+def sgenxyplot(xtype,xmen,ytype,ymen,esettings,
+                    weights_a,weights_b,model_a,model_b,model_c,alpha,beta,mode,calcmode,
+                    useblocks,custom_name,save_sets,id_sets,wpresets,deep,tensor,
+                    prompt,nprompt,steps,sampler,cfg,seed,w,h,
+                    hireson,hrupscaler,hr2ndsteps,denoise_str,hr_scale,batch_size):
+    global hear
+    esettings = " ".join(esettings)
+    #type[0:none,1:aplha,2:beta,3:seed,4:mbw,5:model_A,6:model_B,7:model_C,8:pinpoint 9:deep]
+    xtype = TYPES[xtype]
+    ytype = TYPES[ytype]
+    if ytype == "none": ymen = ""
+
+    modes=["Weight" ,"Add" ,"Triple","Twice"]
+    xs=ys=0
+    weights_a_in=weights_b_in="0"
+
+    deepprint  = True if "print change" in esettings else False
+
+    def castall(hear):
+        if hear :print(f"xmen:{xmen}, ymen:{ymen}, xtype:{xtype}, ytype:{ytype}, weights_a:{weights_a_in}, weights_b:{weights_b_in}, model_A:{model_a},model_B :{model_b}, model_C:{model_c}, alpha:{alpha},\
+        beta :{beta}, mode:{mode}, blocks:{useblocks}")
+
+    pinpoint = "pinpoint blocks" in xtype or "pinpoint blocks" in ytype
+    usebeta = modes[2] in mode or modes[3] in mode
+
+    #check and adjust format
+    print(f"XY plot start, mode:{mode}, X: {xtype}, Y: {ytype}, MBW: {useblocks}")
+    castall(hear)
+    None5 = [None,None,None,None,None]
+    if xmen =="": return "ERROR: parameter X is empty",*None5
+    if ymen =="" and not ytype=="none": return "ERROR: parameter Y is empty",*None5
+    if model_a ==[] and not ("model_A" in xtype or "model_A" in ytype):return f"ERROR: model_A is not selected",*None5
+    if model_b ==[] and not ("model_B" in xtype or "model_B" in ytype):return f"ERROR: model_B is not selected",*None5
+    if model_c ==[] and usebeta and not ("model_C" in xtype or "model_C" in ytype):return "ERROR: model_C is not selected",*None5
+    if xtype == ytype: return "ERROR: same type selected for X,Y",*None5
+
+    if useblocks:
+        weights_a_in=wpreseter(weights_a,wpresets)
+        weights_b_in=wpreseter(weights_b,wpresets)
+
+    #for X only plot, use same seed
+    if seed == -1: seed = int(random.randrange(4294967294))
+
+    #for XY plot, use same seed
+    def dicedealer(zs):
+        for i,z in enumerate(zs):
+            if z =="-1": zs[i] = str(random.randrange(4294967294))
+        print(f"the die was thrown : {zs}")
+
+    #adjust parameters, alpha,beta,models,seed: list of single parameters, mbw(no beta):list of text,mbw(usebeta); list of pair text
+    def adjuster(zmen,ztype,aztype):
+        if "mbw" in ztype or "prompt" in ztype:#men separated by newline
+            zs = zmen.splitlines()
+            caster(zs,hear)
+            if "mbw alpha and beta" in ztype:
+                zs = [zs[i:i+2] for i in range(0,len(zs),2)]
+                caster(zs,hear)
+        elif "elemental" in ztype:
+            zs = zmen.split("\n\n")
+        else:
+            if "pinpoint element" in ztype:
+                zmen = zmen.replace("\n",",")
+            if "effective" in ztype:
+                zmen = ","+zmen
+                zmen = zmen.replace("\n",",")
+            zs = [z.strip() for z in zmen.split(',')]
+            caster(zs,hear)
+        if "alpha" in ztype and "effective" in aztype:
+            zs = [zs[0]]
+        if "seed" in ztype:dicedealer(zs)
+        if "alpha" == ztype or "beta" == ztype:
+            oz = []
+            for z in zs:
+                try:
+                    float(z)
+                    oz.append(z)
+                except:
+                    pass
+            zs = oz
+        return zs
+
+    xs = adjuster(xmen,xtype,ytype)
+    ys = adjuster(ymen,ytype,xtype)
+
+    #in case beta selected but mode is Weight sum or Add or Diff
+    if ("beta" in xtype or "beta" in ytype) and (not usebeta and "tensor" not in calcmode):
+        mode = modes[3]
+        print(f"{modes[3]} mode automatically selected)")
+
+    #in case mbw or pinpoint selected but useblocks not chekced
+    if ("mbw" in xtype or "pinpoint blocks" in xtype) and not useblocks:
+        useblocks = True
+        print(f"MBW mode enabled")
+
+    if ("mbw" in ytype or "pinpoint blocks" in ytype) and not useblocks:
+        useblocks = True
+        print(f"MBW mode enabled")
+
+    xyimage=[]
+    xcount =ycount=0
+    allcount = len(xs)*len(ys)
+
+    #for STOP XY bottun
+    flag = False
+    global state_mergen
+    state_mergen = False
+
+    #type[0:none,1:aplha,2:beta,3:seed,4:mbw,5:model_A,6:model_B,7:model_C,8:pinpoint ]
+    blockid=["BASE","IN00","IN01","IN02","IN03","IN04","IN05","IN06","IN07","IN08","IN09","IN10","IN11","M00","OUT00","OUT01","OUT02","OUT03","OUT04","OUT05","OUT06","OUT07","OUT08","OUT09","OUT10","OUT11"]
+    #format ,IN00 IN03,IN04-IN09,OUT4,OUT05
+    def weightsdealer(x,xtype,y,weights):
+        caster(f"weights from : {weights}",hear)
+        zz = x if "pinpoint blocks" in xtype else y
+        za = y if "pinpoint blocks" in xtype else x
+        zz = [z.strip() for z in zz.split(' ')]
+        weights_t = [w.strip() for w in weights.split(',')]
+        if zz[0]!="NOT":
+            flagger=[False]*26
+            changer = True
+        else:
+            flagger=[True]*26
+            changer = False
+        for z in zz:
+            if z =="NOT":continue
+            if "-" in z:
+                zt = [zt.strip() for zt in z.split('-')]
+                if  blockid.index(zt[1]) > blockid.index(zt[0]):
+                    flagger[blockid.index(zt[0]):blockid.index(zt[1])+1] = [changer]*(blockid.index(zt[1])-blockid.index(zt[0])+1)
+                else:
+                    flagger[blockid.index(zt[1]):blockid.index(zt[0])+1] = [changer]*(blockid.index(zt[0])-blockid.index(zt[1])+1)
+            else:
+                flagger[blockid.index(z)] =changer    
+        for i,f in enumerate(flagger):
+            if f:weights_t[i]=za
+        outext = ",".join(weights_t)
+        caster(f"weights changed: {outext}",hear)
+        return outext
+
+    def abdealer(z):
+        if " " in z:return z.split(" ")[0],z.split(" ")[1]
+        return z,z
+
+    def xydealer(z,zt,azt):
+        nonlocal alpha,beta,seed,weights_a_in,weights_b_in,model_a,model_b,model_c,deep,calcmode,prompt
+        if pinpoint or "pinpoint element" in zt or "effective" in zt:return
+        if "mbw" in zt:
+            def weightser(z):return z, z.split(',',1)[0]
+            if "mbw alpha and beta" in zt:
+                weights_a_in,alpha = weightser(wpreseter(z[0],wpresets))
+                weights_b_in,beta = weightser(wpreseter(z[1],wpresets))
+                return
+            elif "alpha" in zt:
+                weights_a_in,alpha = weightser(wpreseter(z,wpresets))
+                return
+            else:
+                weights_b_in,beta = weightser(wpreseter(z,wpresets))
+                return
+        if "and" in zt:
+            alpha,beta = abdealer(z)
+            return
+        if "alpha" in zt and not "pinpoint element" in azt:alpha = z
+        if "beta" in zt: beta = z
+        if "seed" in zt:seed = int(z)
+        if "model_A" in zt:model_a = z
+        if "model_B" in zt:model_b = z
+        if "model_C" in zt:model_c = z
+        if "elemental" in zt:deep = z
+        if "calcmode" in zt:calcmode = z
+        if "prompt" in zt:prompt = z
+    
+    # plot start
+    for y in ys:
+        xydealer(y,ytype,xtype)
+        xcount = 0
+        for x in xs:
+            xydealer(x,xtype,ytype)
+            if ("alpha" in xtype or "alpha" in ytype) and pinpoint:
+                weights_a_in = weightsdealer(x,xtype,y,weights_a)
+                weights_b_in = weights_b
+            if ("beta" in xtype or "beta" in ytype) and pinpoint:
+                weights_b_in = weightsdealer(x,xtype,y,weights_b)
+                weights_a_in =weights_a
+            if "pinpoint element" in xtype or "effective" in xtype:
+                deep_in = deep +","+ str(x)+":"+ str(y) 
+            elif "pinpoint element" in ytype or "effective" in ytype:
+                deep_in = deep +","+ str(y)+":"+ str(x) 
+            else:
+                deep_in = deep
+
+            print(f"XY plot: X: {xtype}, {str(x)}, Y: {ytype}, {str(y)} ({xcount+ycount*len(xs)+1}/{allcount})")
+            if not (xtype=="seed" and xcount > 0):
+               _ , currentmodel,modelid,theta_0,_=smerge(weights_a_in,weights_b_in, model_a,model_b,model_c, float(alpha),float(beta),mode,calcmode,
+                                                                                useblocks,"","",id_sets,False,deep_in,tensor,deepprint = deepprint) 
+               usemodelgen(theta_0,model_a,currentmodel)
+                             # simggen(prompt, nprompt, steps, sampler, cfg, seed, w, h,mergeinfo="",id_sets=[],modelid = "no id"):
+            image_temp = simggen(prompt, nprompt, steps, sampler, cfg, seed, w, h,hireson,hrupscaler,hr2ndsteps,denoise_str,hr_scale,batch_size,currentmodel,id_sets,modelid)
+            xyimage.append(image_temp[0][0])
+            xcount+=1
+            if state_mergen:
+                flag = True
+                break
+        ycount+=1
+        if flag:break
+
+    if flag and ycount ==1:
+        xs = xs[:xcount]
+        ys = [ys[0],]
+        print(f"stopped at x={xcount},y={ycount}")
+    elif flag:
+        ys=ys[:ycount]
+        print(f"stopped at x={xcount},y={ycount}")
+
+    if "mbw alpha and beta" in xtype: xs = [f"alpha:({x[0]}),beta({x[1]})" for x in xs ]
+    if "mbw alpha and beta" in ytype: ys = [f"alpha:({y[0]}),beta({y[1]})" for y in ys ]
+
+    xs[0]=xtype+" = "+xs[0] #draw X label
+    if ytype!=TYPES[0] or "model" in ytype:ys[0]=ytype+" = "+ys[0]  #draw Y label
+
+    if ys==[""]:ys = [" "]
+
+    if "effective" in xtype or "effective" in ytype:
+        xyimage,xs,ys = effectivechecker(xyimage,xs,ys,model_a,model_b,esettings)
+
+    if not "grid" in esettings:
+        gridmodel= makegridmodelname(model_a, model_b,model_c, useblocks,mode,xtype,ytype,alpha,beta,weights_a,weights_b,usebeta)
+        grid = smakegrid(xyimage,xs,ys,gridmodel,image_temp[4])
+        xyimage.insert(0,grid)
+
+    state_mergen = False
+    return "Finished",currentmodel,xyimage,*image_temp[1:4]
+
+def smakegrid(imgs,xs,ys,currentmodel,p):
+    ver_texts = [[images.GridAnnotation(y)] for y in ys]
+    hor_texts = [[images.GridAnnotation(x)] for x in xs]
+
+    w, h = imgs[0].size
+    grid = Image.new('RGB', size=(len(xs) * w, len(ys) * h), color='black')
+
+    for i, img in enumerate(imgs):
+        grid.paste(img, box=(i % len(xs) * w, i // len(xs) * h))
+
+    grid = images.draw_grid_annotations(grid,w,h, hor_texts, ver_texts)
+    grid = draw_origin(grid, currentmodel,w*len(xs),h*len(ys),w)
+    if opts.grid_save:
+        images.save_image(grid, opts.outdir_txt2img_grids, "xy_grid", extension=opts.grid_format, prompt=p.prompt, seed=p.seed, grid=True, p=p)
+
+    return grid
+
+def makegridmodelname(model_a, model_b,model_c, useblocks,mode,xtype,ytype,alpha,beta,wa,wb,usebeta):
+    model_a=filenamecutter(model_a)
+    model_b=filenamecutter(model_b)
+    model_c=filenamecutter(model_c)
+
+    if not usebeta:beta,wb = "not used","not used"
+    vals = ""
+    modes=["Weight" ,"Add" ,"Triple","Twice"]
+
+    if "mbw" in xtype:
+        if "alpha" in xtype:wa = "X"
+        if usebeta or " beta" in xtype:wb = "X"
+
+    if "mbw" in ytype:
+        if "alpha" in ytype:wa = "Y"
+        if usebeta or " beta" in ytype:wb = "Y"
+
+    wa = "alpha = " + wa
+    wb = "beta = " + wb
+
+    x = 50
+    while len(wa) > x:
+        wa  = wa[:x] + '\n' + wa[x:]
+        x = x + 50
+
+    x = 50
+    while len(wb) > x:
+        wb  = wb[:x] + '\n' + wb[x:]
+        x = x + 50
+
+    if "model" in xtype:
+        if "A" in xtype:model_a = "model A"
+        elif "B" in xtype:model_b="model B"
+        elif "C" in xtype:model_c="model C"
+
+    if "model" in ytype:
+        if "A" in ytype:model_a = "model A"
+        elif "B" in ytype:model_b="model B"
+        elif "C" in ytype:model_c="model C"
+
+    if modes[1] in mode:
+        currentmodel =f"{model_a} \n {model_b} - {model_c})\n x alpha"
+    elif modes[2] in mode:
+        currentmodel =f"{model_a} x \n(1-alpha-beta) {model_b} x alpha \n+ {model_c} x beta"
+    elif modes[3] in mode:
+        currentmodel =f"({model_a} x(1-alpha) \n + {model_b} x alpha)*(1-beta)\n+  {model_c} x beta"
+    else:
+        currentmodel =f"{model_a} x (1-alpha) \n {model_b} x alpha"
+
+    if "alpha" in xtype:alpha = "X"
+    if "beta" in xtype:beta = "X" 
+    if "alpha" in ytype:alpha = "Y"
+    if "beta" in ytype:beta = "Y"
+
+    if "mbw" in xtype:
+        if "alpha" in xtype: alpha = "X"
+        if "beta" in xtype or usebeta: beta = "X"
+
+    if "mbw" in ytype:
+        if "alpha" in ytype: alpha = "Y"
+        if "beta" in ytype or usebeta: beta = "Y"
+
+    vals = f"\nalpha = {alpha},beta = {beta}" if not useblocks else f"\n{wa}\n{wb}"
+
+    currentmodel = currentmodel+vals
+    return currentmodel
+
+def effectivechecker(imgs,xs,ys,model_a,model_b,esettings):
+    diffs = []
+    outnum =[]
+    im1 = np.array(imgs[0])
+    
+    model_a = filenamecutter(model_a)
+    model_b = filenamecutter(model_b)
+    dir = os.path.join(opts.outdir_txt2img_samples,f"{model_a+model_b}","difgif")
+
+    if "gif" in esettings:
+        try:
+            os.makedirs(dir)
+        except FileExistsError:
+            pass
+
+    ls,ss = (xs.copy(),ys.copy()) if len(xs) > len(ys) else (ys.copy(),xs.copy())
+
+    for i in range(len(imgs)-1):
+        im2 = np.array(imgs[i+1])
+
+        abs_diff = cv2.absdiff(im2 ,  im1)
+
+        abs_diff_t = cv2.threshold(abs_diff, 5, 255, cv2.THRESH_BINARY)[1]        
+        res = abs_diff_t.astype(np.uint8)
+        percentage = (np.count_nonzero(res) * 100)/ res.size
+        abs_diff = cv2.bitwise_not(abs_diff)
+        outnum.append(percentage)
+
+        abs_diff = Image.fromarray(abs_diff)     
+
+        diffs.append(abs_diff)
+
+        if "gif" in esettings:
+            gifpath = gifpath_t = os.path.join(dir,ls[i+1].replace(":","_")+".gif")
+            
+            is_file = os.path.isfile(gifpath)
+            j = 0
+            while is_file:
+                gifpath = gifpath_t.replace(".gif",f"_{j}.gif")
+                print(gifpath)
+                is_file = os.path.isfile(gifpath)
+                j = j + 1
+
+            imgs[0].save(gifpath, save_all=True, append_images=[imgs[i+1]], optimize=False, duration=1000, loop=0)
+
+    nums = []
+    outs = []
+
+    ls = ls[1:]
+    for i in range(len(ls)):
+        nums.append([ls[i],outnum[i]])
+        ls[i] = ls[i] + "\n Diff : " + str(round(outnum[i],3)) + "%"    
+
+    if "csv" in esettings:
+        try:
+            os.makedirs(dir)
+        except FileExistsError:
+            pass
+        filepath = os.path.join(dir, f"{model_a+model_b}.csv")
+        with open(filepath, "a", newline="") as f:
+            writer = csv.writer(f)
+            writer.writerows(nums)
+
+    if len(ys) > len (xs):
+        for diff,img in zip(diffs,imgs[1:]):
+            outs.append(diff)
+            outs.append(img)
+            outs.append(imgs[0])
+        ss = ["diff",ss[0],"source"]
+        return outs,ss,ls
+    else:
+        outs = [imgs[0]]*len(diffs)  + imgs[1:]+ diffs
+        ss = ["source",ss[0],"diff"]
+        return outs,ls,ss

microsoftexcel-supermerger/scripts/supermerger.py

@@ -0,0 +1,552 @@
+import argparse
+import gc
+import os
+import os.path
+import re
+import shutil
+from importlib import reload
+from pprint import pprint
+import gradio as gr
+from modules import (devices, script_callbacks, scripts, sd_hijack, sd_models,sd_vae, shared)
+from modules.scripts import basedir
+from modules.sd_models import checkpoints_loaded
+from modules.shared import opts
+from modules.ui import create_output_panel, create_refresh_button
+import scripts.mergers.mergers
+import scripts.mergers.pluslora
+import scripts.mergers.xyplot
+reload(scripts.mergers.mergers) # update without restarting web-ui.bat
+reload(scripts.mergers.xyplot)
+reload(scripts.mergers.pluslora)
+import csv
+import scripts.mergers.pluslora as pluslora
+from scripts.mergers.mergers import (TYPESEG, freezemtime, rwmergelog, simggen,smergegen)
+from scripts.mergers.xyplot import freezetime, nulister, numaker, numanager
+
+gensets=argparse.Namespace()
+
+def on_ui_train_tabs(params):
+    txt2img_preview_params=params.txt2img_preview_params
+    gensets.txt2img_preview_params=txt2img_preview_params
+    return None
+
+path_root = basedir()
+
+def on_ui_tabs():
+    weights_presets=""
+    userfilepath = os.path.join(path_root, "scripts","mbwpresets.txt")
+    if os.path.isfile(userfilepath):
+        try:
+            with open(userfilepath) as f:
+                weights_presets = f.read()
+                filepath = userfilepath
+        except OSError as e:
+                pass
+    else:
+        filepath = os.path.join(path_root, "scripts","mbwpresets_master.txt")
+        try:
+            with open(filepath) as f:
+                weights_presets = f.read()
+                shutil.copyfile(filepath, userfilepath)
+        except OSError as e:
+                pass
+
+    with gr.Blocks() as supermergerui:
+        with gr.Tab("Merge"):
+            with gr.Row().style(equal_height=False):
+                with gr.Column(scale = 3):
+                    gr.HTML(value="<p>Merge models and load it for generation</p>")
+
+                    with gr.Row():
+                        model_a = gr.Dropdown(sd_models.checkpoint_tiles(),elem_id="model_converter_model_name",label="Model A",interactive=True)
+                        create_refresh_button(model_a, sd_models.list_models,lambda: {"choices": sd_models.checkpoint_tiles()},"refresh_checkpoint_Z")
+
+                        model_b = gr.Dropdown(sd_models.checkpoint_tiles(),elem_id="model_converter_model_name",label="Model B",interactive=True)
+                        create_refresh_button(model_b, sd_models.list_models,lambda: {"choices": sd_models.checkpoint_tiles()},"refresh_checkpoint_Z")
+
+                        model_c = gr.Dropdown(sd_models.checkpoint_tiles(),elem_id="model_converter_model_name",label="Model C",interactive=True)
+                        create_refresh_button(model_c, sd_models.list_models,lambda: {"choices": sd_models.checkpoint_tiles()},"refresh_checkpoint_Z")
+
+                    mode = gr.Radio(label = "Merge Mode",choices = ["Weight sum:A*(1-alpha)+B*alpha", "Add difference:A+(B-C)*alpha",
+                                                        "Triple sum:A*(1-alpha-beta)+B*alpha+C*beta",
+                                                        "sum Twice:(A*(1-alpha)+B*alpha)*(1-beta)+C*beta",
+                                                         ], value = "Weight sum:A*(1-alpha)+B*alpha") 
+                    calcmode = gr.Radio(label = "Calcutation Mode",choices = ["normal", "cosineA", "cosineB", "smoothAdd","tensor"], value = "normal") 
+                    with gr.Row(): 
+                        useblocks =  gr.Checkbox(label="use MBW")
+                        base_alpha = gr.Slider(label="alpha", minimum=-1.0, maximum=2, step=0.001, value=0.5)
+                        base_beta = gr.Slider(label="beta", minimum=-1.0, maximum=2, step=0.001, value=0.25)
+                        #weights = gr.Textbox(label="weights,base alpha,IN00,IN02,...IN11,M00,OUT00,...,OUT11",lines=2,value="0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5")
+
+                    with gr.Row():
+                        merge = gr.Button(elem_id="model_merger_merge", value="Merge!",variant='primary')
+                        mergeandgen = gr.Button(elem_id="model_merger_merge", value="Merge&Gen",variant='primary')
+                        gen = gr.Button(elem_id="model_merger_merge", value="Gen",variant='primary')
+                        stopmerge = gr.Button(elem_id="stopmerge", value="Stop",variant='primary')
+                    with gr.Row():
+                        with gr.Column(scale = 4):
+                            save_sets = gr.CheckboxGroup(["save model", "overwrite","safetensors","fp16","save metadata"], value=["safetensors"], label="save settings")
+                        with gr.Column(scale = 2):
+                            id_sets = gr.CheckboxGroup(["image", "PNG info"], label="write merged model ID to")
+                    with gr.Row():      
+                        with gr.Column(min_width = 50, scale=2):
+                            with gr.Row():
+                                custom_name = gr.Textbox(label="Custom Name (Optional)", elem_id="model_converter_custom_name")
+                                mergeid = gr.Textbox(label="merge from ID", elem_id="model_converter_custom_name",value = "-1")
+                        with gr.Column(min_width = 50, scale=1):
+                            with gr.Row():s_reverse= gr.Button(value="Set from ID(-1 for last)",variant='primary')
+
+                    with gr.Accordion("Restore faces, Tiling, Hires. fix, Batch size",open = False):
+                        batch_size = denois_str = gr.Slider(minimum=0, maximum=8, step=1, label='Batch size', value=1, elem_id="sm_txt2img_batch_size")
+                        genoptions = gr.CheckboxGroup(label = "Gen Options",choices=["Restore faces", "Tiling", "Hires. fix"], visible = True,interactive=True,type="value")    
+                        with gr.Row(elem_id="txt2img_hires_fix_row1", variant="compact"):
+                            hrupscaler = gr.Dropdown(label="Upscaler", elem_id="txt2img_hr_upscaler", choices=[*shared.latent_upscale_modes, *[x.name for x in shared.sd_upscalers]], value=shared.latent_upscale_default_mode)
+                            hr2ndsteps = gr.Slider(minimum=0, maximum=150, step=1, label='Hires steps', value=0, elem_id="txt2img_hires_steps")
+                            denois_str = gr.Slider(minimum=0.0, maximum=1.0, step=0.01, label='Denoising strength', value=0.7, elem_id="txt2img_denoising_strength")
+                            hr_scale = gr.Slider(minimum=1.0, maximum=4.0, step=0.05, label="Upscale by", value=2.0, elem_id="txt2img_hr_scale")
+                            
+                    hiresfix = [genoptions,hrupscaler,hr2ndsteps,denois_str,hr_scale]
+
+                    with gr.Accordion("Elemental Merge",open = False):
+                        with gr.Row():
+                            esettings1 = gr.CheckboxGroup(label = "settings",choices=["print change"],type="value",interactive=True)
+                        with gr.Row():
+                            deep = gr.Textbox(label="Blocks:Element:Ratio,Blocks:Element:Ratio,...",lines=2,value="")
+                    
+                    with gr.Accordion("Tensor Merge",open = False,visible=False):
+                        tensor = gr.Textbox(label="Blocks:Tensors",lines=2,value="")
+                    
+                    with gr.Row():
+                        x_type = gr.Dropdown(label="X type", choices=[x for x in TYPESEG], value="alpha", type="index")
+                        x_randseednum = gr.Number(value=3, label="number of -1", interactive=True, visible = True)
+                    xgrid = gr.Textbox(label="Sequential Merge Parameters",lines=3,value="0.25,0.5,0.75")
+                    y_type = gr.Dropdown(label="Y type", choices=[y for y in TYPESEG], value="none", type="index")    
+                    ygrid = gr.Textbox(label="Y grid (Disabled if blank)",lines=3,value="",visible =False)
+                    with gr.Row():
+                        gengrid = gr.Button(elem_id="model_merger_merge", value="Sequential XY Merge and Generation",variant='primary')
+                        stopgrid = gr.Button(elem_id="model_merger_merge", value="Stop XY",variant='primary')
+                        s_reserve1 = gr.Button(value="Reserve XY Plot",variant='primary')
+                    dtrue =  gr.Checkbox(value = True, visible = False)                
+                    dfalse =  gr.Checkbox(value = False,visible = False)     
+                    dummy_t =  gr.Textbox(value = "",visible = False)    
+                blockid=["BASE","IN00","IN01","IN02","IN03","IN04","IN05","IN06","IN07","IN08","IN09","IN10","IN11","M00","OUT00","OUT01","OUT02","OUT03","OUT04","OUT05","OUT06","OUT07","OUT08","OUT09","OUT10","OUT11"]
+        
+                with gr.Column(scale = 2):
+                    currentmodel = gr.Textbox(label="Current Model",lines=1,value="")  
+                    submit_result = gr.Textbox(label="Message")
+                    mgallery, mgeninfo, mhtmlinfo, mhtmllog = create_output_panel("txt2img", opts.outdir_txt2img_samples)
+            with gr.Row(visible = False) as row_inputers:
+                inputer = gr.Textbox(label="",lines=1,value="")
+                addtox = gr.Button(value="Add to Sequence X")
+                addtoy = gr.Button(value="Add to Sequence Y")
+            with gr.Row(visible = False) as row_blockids:
+                blockids = gr.CheckboxGroup(label = "block IDs",choices=[x for x in blockid],type="value",interactive=True)
+            with gr.Row(visible = False) as row_calcmode:
+                calcmodes = gr.CheckboxGroup(label = "calcmode",choices=["normal", "cosineA", "cosineB", "smoothAdd","tensor"],type="value",interactive=True)
+            with gr.Row(visible = False) as row_checkpoints:
+                checkpoints = gr.CheckboxGroup(label = "checkpoint",choices=[x.model_name for x in sd_models.checkpoints_list.values()],type="value",interactive=True)
+            with gr.Row(visible = False) as row_esets:
+                esettings = gr.CheckboxGroup(label = "effective chekcer settings",choices=["save csv","save anime gif","not save grid","print change"],type="value",interactive=True)
+    
+            with gr.Tab("Weights Setting"):
+                with gr.Row():
+                    setalpha = gr.Button(elem_id="copytogen", value="set to alpha",variant='primary')
+                    readalpha = gr.Button(elem_id="copytogen", value="read from alpha",variant='primary')
+                    setbeta = gr.Button(elem_id="copytogen", value="set to beta",variant='primary')
+                    readbeta = gr.Button(elem_id="copytogen", value="read from beta",variant='primary')
+                    setx = gr.Button(elem_id="copytogen", value="set to X",variant='primary')
+                with gr.Row():
+                    weights_a = gr.Textbox(label="weights for alpha, base alpha,IN00,IN02,...IN11,M00,OUT00,...,OUT11",value = "0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5")
+                    weights_b = gr.Textbox(label="weights,for beta, base beta,IN00,IN02,...IN11,M00,OUT00,...,OUT11",value = "0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2")
+                with gr.Row():
+                    base= gr.Slider(label="Base", minimum=0, maximum=1, step =0.01, value=0.5)
+                    in00 = gr.Slider(label="IN00", minimum=0, maximum=1, step=0.01, value=0.5)
+                    in01 = gr.Slider(label="IN01", minimum=0, maximum=1, step=0.01, value=0.5)
+                    in02 = gr.Slider(label="IN02", minimum=0, maximum=1, step=0.01, value=0.5)
+                    in03 = gr.Slider(label="IN03", minimum=0, maximum=1, step=0.01, value=0.5)
+                with gr.Row():
+                    in04 = gr.Slider(label="IN04", minimum=0, maximum=1, step=0.01, value=0.5)
+                    in05 = gr.Slider(label="IN05", minimum=0, maximum=1, step=0.01, value=0.5)
+                    in06 = gr.Slider(label="IN06", minimum=0, maximum=1, step=0.01, value=0.5)
+                    in07 = gr.Slider(label="IN07", minimum=0, maximum=1, step=0.01, value=0.5)
+                    in08 = gr.Slider(label="IN08", minimum=0, maximum=1, step=0.01, value=0.5)
+                    in09 = gr.Slider(label="IN09", minimum=0, maximum=1, step=0.01, value=0.5)
+                with gr.Row():
+                    in10 = gr.Slider(label="IN10", minimum=0, maximum=1, step=0.01, value=0.5)
+                    in11 = gr.Slider(label="IN11", minimum=0, maximum=1, step=0.01, value=0.5)
+                    mi00 = gr.Slider(label="M00", minimum=0, maximum=1, step=0.01, value=0.5)
+                    ou00 = gr.Slider(label="OUT00", minimum=0, maximum=1, step=0.01, value=0.5)
+                    ou01 = gr.Slider(label="OUT01", minimum=0, maximum=1, step=0.01, value=0.5)
+                    ou02 = gr.Slider(label="OUT02", minimum=0, maximum=1, step=0.01, value=0.5)
+                with gr.Row():
+                    ou03 = gr.Slider(label="OUT03", minimum=0, maximum=1, step=0.01, value=0.5)
+                    ou04 = gr.Slider(label="OUT04", minimum=0, maximum=1, step=0.01, value=0.5)
+                    ou05 = gr.Slider(label="OUT05", minimum=0, maximum=1, step=0.01, value=0.5)
+                    ou06 = gr.Slider(label="OUT06", minimum=0, maximum=1, step=0.01, value=0.5)
+                    ou07 = gr.Slider(label="OUT07", minimum=0, maximum=1, step=0.01, value=0.5)
+                    ou08 = gr.Slider(label="OUT08", minimum=0, maximum=1, step=0.01, value=0.5)
+                with gr.Row(): 
+                    ou09 = gr.Slider(label="OUT09", minimum=0, maximum=1, step=0.01, value=0.5)       
+                    ou10 = gr.Slider(label="OUT10", minimum=0, maximum=1, step=0.01, value=0.5)
+                    ou11 = gr.Slider(label="OUT11", minimum=0, maximum=1, step=0.01, value=0.5)
+            with gr.Tab("Weights Presets"):
+                with gr.Row():
+                    s_reloadtext = gr.Button(value="Reload Presets",variant='primary')
+                    s_reloadtags = gr.Button(value="Reload Tags",variant='primary')
+                    s_savetext = gr.Button(value="Save Presets",variant='primary')
+                    s_openeditor = gr.Button(value="Open TextEditor",variant='primary')
+                weightstags= gr.Textbox(label="available",lines = 2,value=tagdicter(weights_presets),visible =True,interactive =True) 
+                wpresets= gr.TextArea(label="",value=weights_presets,visible =True,interactive  = True)    
+
+            with gr.Tab("Reservation"):
+                with gr.Row():
+                    s_reserve = gr.Button(value="Reserve XY Plot",variant='primary')
+                    s_reloadreserve = gr.Button(value="Reloat List",variant='primary')
+                    s_startreserve = gr.Button(value="Start XY plot",variant='primary')
+                    s_delreserve = gr.Button(value="Delete list(-1 for all)",variant='primary')
+                    s_delnum = gr.Number(value=1, label="Delete num : ", interactive=True, visible = True,precision =0)
+                with gr.Row():
+                    numaframe = gr.Dataframe(
+                        headers=["No.","status","xtype","xmenber", "ytype","ymenber","model A","model B","model C","alpha","beta","mode","use MBW","weights alpha","weights beta"],
+                        row_count=5,)
+            # with gr.Tab("manual"):
+            #     with gr.Row():
+            #         gr.HTML(value="<p> exampls: Change base alpha from 0.1 to 0.9 <br>0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9<br>If you want to display the original model as well for comparison<br>0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1</p>")
+            #         gr.HTML(value="<p> For block-by-block merging <br>0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5<br>1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1<br>0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1</p>")
+            
+            with gr.Row():
+
+                currentcache = gr.Textbox(label="Current Cache")
+                loadcachelist = gr.Button(elem_id="model_merger_merge", value="Reload Cache List",variant='primary')
+                unloadmodel = gr.Button(value="unload model",variant='primary')
+
+
+        # main ui end 
+    
+        with gr.Tab("LoRA", elem_id="tab_lora"):
+            pluslora.on_ui_tabs()
+
+        with gr.Tab("History", elem_id="tab_history"):
+            
+            with gr.Row():
+                load_history = gr.Button(value="load_history",variant='primary')
+                searchwrods = gr.Textbox(label="",lines=1,value="")
+                search = gr.Button(value="search")
+                searchmode = gr.Radio(label = "Search Mode",choices = ["or","and"], value = "or",type  = "value") 
+            with gr.Row():
+                history = gr.Dataframe(
+                        headers=["ID","Time","Name","Weights alpha","Weights beta","Model A","Model B","Model C","alpha","beta","Mode","use MBW","custum name","save setting","use ID"],
+                )
+
+        with gr.Tab("Elements", elem_id="tab_deep"):
+                with gr.Row():
+                    smd_model_a = gr.Dropdown(sd_models.checkpoint_tiles(),elem_id="model_converter_model_name",label="Checkpoint A",interactive=True)
+                    create_refresh_button(smd_model_a, sd_models.list_models,lambda: {"choices": sd_models.checkpoint_tiles()},"refresh_checkpoint_Z")    
+                    smd_loadkeys = gr.Button(value="load keys",variant='primary')
+                with gr.Row():
+                    keys = gr.Dataframe(headers=["No.","block","key"],)
+
+        with gr.Tab("Metadeta", elem_id="tab_metadata"):
+                with gr.Row():
+                    meta_model_a = gr.Dropdown(sd_models.checkpoint_tiles(),elem_id="model_converter_model_name",label="read metadata",interactive=True)
+                    create_refresh_button(meta_model_a, sd_models.list_models,lambda: {"choices": sd_models.checkpoint_tiles()},"refresh_checkpoint_Z")    
+                    smd_loadmetadata = gr.Button(value="load keys",variant='primary')
+                with gr.Row():
+                    metadata = gr.TextArea()
+
+        smd_loadmetadata.click(
+            fn=loadmetadata,
+            inputs=[meta_model_a],
+            outputs=[metadata]
+        )                 
+
+        smd_loadkeys.click(
+            fn=loadkeys,
+            inputs=[smd_model_a],
+            outputs=[keys]
+        )
+
+        def unload():
+            if shared.sd_model == None: return "already unloaded"
+            sd_hijack.model_hijack.undo_hijack(shared.sd_model)
+            shared.sd_model = None
+            gc.collect()
+            devices.torch_gc()
+            return "model unloaded"
+
+        unloadmodel.click(fn=unload,outputs=[submit_result])
+
+        load_history.click(fn=load_historyf,outputs=[history ])
+
+        msettings=[weights_a,weights_b,model_a,model_b,model_c,base_alpha,base_beta,mode,calcmode,useblocks,custom_name,save_sets,id_sets,wpresets,deep,tensor]
+        imagegal = [mgallery,mgeninfo,mhtmlinfo,mhtmllog]
+        xysettings=[x_type,xgrid,y_type,ygrid,esettings]
+
+        s_reverse.click(fn = reversparams,
+            inputs =mergeid,
+            outputs = [submit_result,*msettings[0:8],*msettings[9:13],deep,calcmode]
+        )
+
+        merge.click(
+            fn=smergegen,
+            inputs=[*msettings,esettings1,*gensets.txt2img_preview_params,*hiresfix,batch_size,currentmodel,dfalse],
+            outputs=[submit_result,currentmodel]
+        )
+
+        mergeandgen.click(
+            fn=smergegen,
+            inputs=[*msettings,esettings1,*gensets.txt2img_preview_params,*hiresfix,batch_size,currentmodel,dtrue],
+            outputs=[submit_result,currentmodel,*imagegal]
+        )
+
+        gen.click(
+            fn=simggen,
+            inputs=[*gensets.txt2img_preview_params,*hiresfix,batch_size,currentmodel,id_sets],
+            outputs=[*imagegal],
+        )
+
+        s_reserve.click(
+            fn=numaker,
+            inputs=[*xysettings,*msettings,*gensets.txt2img_preview_params,*hiresfix,batch_size],
+            outputs=[numaframe]
+        )
+
+        s_reserve1.click(
+            fn=numaker,
+            inputs=[*xysettings,*msettings,*gensets.txt2img_preview_params,*hiresfix,batch_size],
+            outputs=[numaframe]
+        )
+
+        gengrid.click(
+            fn=numanager,
+            inputs=[dtrue,*xysettings,*msettings,*gensets.txt2img_preview_params,*hiresfix,batch_size],
+            outputs=[submit_result,currentmodel,*imagegal],
+        )
+
+        s_startreserve.click(
+            fn=numanager,
+            inputs=[dfalse,*xysettings,*msettings,*gensets.txt2img_preview_params,*hiresfix,batch_size],
+            outputs=[submit_result,currentmodel,*imagegal],
+        )
+
+        search.click(fn = searchhistory,inputs=[searchwrods,searchmode],outputs=[history])
+
+        s_reloadreserve.click(fn=nulister,inputs=[dfalse],outputs=[numaframe])
+        s_delreserve.click(fn=nulister,inputs=[s_delnum],outputs=[numaframe])
+        loadcachelist.click(fn=load_cachelist,inputs=[],outputs=[currentcache])
+        addtox.click(fn=lambda x:gr.Textbox.update(value = x),inputs=[inputer],outputs=[xgrid])
+        addtoy.click(fn=lambda x:gr.Textbox.update(value = x),inputs=[inputer],outputs=[ygrid])
+
+        stopgrid.click(fn=freezetime)
+        stopmerge.click(fn=freezemtime)
+
+        checkpoints.change(fn=lambda x:",".join(x),inputs=[checkpoints],outputs=[inputer])
+        blockids.change(fn=lambda x:" ".join(x),inputs=[blockids],outputs=[inputer])
+        calcmodes.change(fn=lambda x:",".join(x),inputs=[calcmodes],outputs=[inputer])
+
+        menbers = [base,in00,in01,in02,in03,in04,in05,in06,in07,in08,in09,in10,in11,mi00,ou00,ou01,ou02,ou03,ou04,ou05,ou06,ou07,ou08,ou09,ou10,ou11]
+
+        setalpha.click(fn=slider2text,inputs=menbers,outputs=[weights_a])
+        setbeta.click(fn=slider2text,inputs=menbers,outputs=[weights_b])
+        setx.click(fn=add_to_seq,inputs=[xgrid,weights_a],outputs=[xgrid])     
+
+        readalpha.click(fn=text2slider,inputs=weights_a,outputs=menbers)
+        readbeta.click(fn=text2slider,inputs=weights_b,outputs=menbers)
+
+        x_type.change(fn=showxy,inputs=[x_type,y_type], outputs=[row_blockids,row_checkpoints,row_inputers,ygrid,row_esets,row_calcmode])
+        y_type.change(fn=showxy,inputs=[x_type,y_type], outputs=[row_blockids,row_checkpoints,row_inputers,ygrid,row_esets,row_calcmode])
+        x_randseednum.change(fn=makerand,inputs=[x_randseednum],outputs=[xgrid])
+
+        import subprocess
+        def openeditors():
+            subprocess.Popen(['start', filepath], shell=True)
+
+        def reloadpresets():
+            try:
+                with open(filepath) as f:
+                    return f.read()
+            except OSError as e:
+                pass
+
+        def savepresets(text):
+            with open(filepath,mode = 'w') as f:
+                f.write(text)
+
+        s_reloadtext.click(fn=reloadpresets,inputs=[],outputs=[wpresets])
+        s_reloadtags.click(fn=tagdicter,inputs=[wpresets],outputs=[weightstags])
+        s_savetext.click(fn=savepresets,inputs=[wpresets],outputs=[])
+        s_openeditor.click(fn=openeditors,inputs=[],outputs=[])
+
+    return (supermergerui, "SuperMerger", "supermerger"),
+
+msearch = []
+mlist=[]
+
+def loadmetadata(model):
+    import json
+    checkpoint_info = sd_models.get_closet_checkpoint_match(model)
+    if ".safetensors" not in checkpoint_info.filename: return "no metadata(not safetensors)"
+    sdict = sd_models.read_metadata_from_safetensors(checkpoint_info.filename)
+    if sdict == {}: return "no metadata"
+    return json.dumps(sdict,indent=4)
+
+def load_historyf():
+    filepath = os.path.join(path_root,"mergehistory.csv")
+    global mlist,msearch
+    msearch = []
+    mlist=[]
+    try:
+        with  open(filepath, 'r') as f:
+            reader = csv.reader(f)
+            mlist =  [raw for raw in reader]
+            mlist = mlist[1:]
+            for m in mlist:
+                msearch.append(" ".join(m))
+            maxlen = len(mlist[-1][0])
+            for i,m in enumerate(mlist):
+                mlist[i][0] = mlist[i][0].zfill(maxlen)
+            return mlist
+    except:
+        return [["no data","",""],]
+
+def searchhistory(words,searchmode):
+    outs =[]
+    ando = "and" in searchmode
+    words = words.split(" ") if " " in words else [words]
+    for i, m in  enumerate(msearch):
+        hit = ando
+        for w in words:
+            if ando:
+                if w not in m:hit = False
+            else:
+                if w in m:hit = True
+        print(i,len(mlist))
+        if hit :outs.append(mlist[i])
+
+    if outs == []:return [["no result","",""],]
+    return outs
+
+#msettings=[0 weights_a,1 weights_b,2 model_a,3 model_b,4 model_c,5 base_alpha,6 base_beta,7 mode,8 useblocks,9 custom_name,10 save_sets,11 id_sets,12 wpresets]
+
+def reversparams(id):
+    def selectfromhash(hash):
+        for model in sd_models.checkpoint_tiles():
+            if hash in model:
+                return model
+        return ""
+    try:
+        idsets = rwmergelog(id = id)
+    except:
+        return [gr.update(value = "ERROR: history file could not open"),*[gr.update() for x in range(14)]]
+    if type(idsets) == str:
+        print("ERROR")
+        return [gr.update(value = idsets),*[gr.update() for x in range(14)]]
+    if idsets[0] == "ID":return  [gr.update(value ="ERROR: no history"),*[gr.update() for x in range(14)]]
+    mgs = idsets[3:]
+    if mgs[0] == "":mgs[0] = "0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5"
+    if mgs[1] == "":mgs[1] = "0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2"
+    mgs[2] = selectfromhash(mgs[2]) if len(mgs[2]) > 5 else ""
+    mgs[3] = selectfromhash(mgs[3]) if len(mgs[3]) > 5 else ""
+    mgs[4] = selectfromhash(mgs[4]) if len(mgs[4]) > 5 else ""
+    mgs[8] = True if mgs[8] =="True" else False
+    mgs[10] = mgs[10].replace("[","").replace("]","").replace("'", "") 
+    mgs[10] = [x.strip() for x in mgs[10].split(",")]
+    mgs[11] = mgs[11].replace("[","").replace("]","").replace("'", "") 
+    mgs[11] = [x.strip() for x in mgs[11].split(",")]
+    while len(mgs) < 14:
+        mgs.append("")
+    mgs[13] = "normal" if mgs[13] == "" else mgs[13] 
+    return [gr.update(value = "setting loaded") ,*[gr.update(value = x) for x in mgs[0:14]]]
+
+def add_to_seq(seq,maker):
+    return gr.Textbox.update(value = maker if seq=="" else seq+"\r\n"+maker)
+
+def load_cachelist():
+    text = ""
+    for x in checkpoints_loaded.keys():
+        text = text +"\r\n"+ x.model_name
+    return text.replace("\r\n","",1)
+
+def makerand(num):
+    text = ""
+    for x in range(int(num)):
+        text = text +"-1,"
+    text = text[:-1]
+    return text
+
+#row_blockids,row_checkpoints,row_inputers,ygrid
+def showxy(x,y):
+    flags =[False]*6
+    t = TYPESEG
+    txy = t[x] + t[y]
+    if "model" in txy : flags[1] = flags[2] = True
+    if "pinpoint" in txy : flags[0] = flags[2] = True
+    if "effective" in txy or "element" in txy : flags[4] = True
+    if "calcmode" in txy : flags[5] = True
+    if not "none" in t[y] : flags[3] = flags[2] = True
+    return [gr.update(visible = x) for x in flags]
+
+def text2slider(text):
+    vals = [t.strip() for t in text.split(",")]
+    return [gr.update(value = float(v)) for v in vals]
+
+def slider2text(a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z):
+    numbers = [a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z]
+    numbers = [str(x) for x in numbers]
+    return gr.update(value = ",".join(numbers) )
+
+def tagdicter(presets):
+    presets=presets.splitlines()
+    wdict={}
+    for l in presets:
+        w=[]
+        if ":" in l :
+            key = l.split(":",1)[0]
+            w = l.split(":",1)[1]
+        if "\t" in l:
+            key = l.split("\t",1)[0]
+            w = l.split("\t",1)[1]
+        if len([w for w in w.split(",")]) == 26:
+            wdict[key.strip()]=w
+    return ",".join(list(wdict.keys()))
+
+def loadkeys(model_a):
+    checkpoint_info = sd_models.get_closet_checkpoint_match(model_a)
+    sd = sd_models.read_state_dict(checkpoint_info.filename,"cpu")
+    keys = []
+    for i, key in enumerate(sd.keys()):
+        re_inp = re.compile(r'\.input_blocks\.(\d+)\.')  # 12
+        re_mid = re.compile(r'\.middle_block\.(\d+)\.')  # 1
+        re_out = re.compile(r'\.output_blocks\.(\d+)\.') # 12
+
+        weight_index = -1
+        blockid=["BASE","IN00","IN01","IN02","IN03","IN04","IN05","IN06","IN07","IN08","IN09","IN10","IN11","M00","OUT00","OUT01","OUT02","OUT03","OUT04","OUT05","OUT06","OUT07","OUT08","OUT09","OUT10","OUT11","Not Merge"]
+
+        NUM_INPUT_BLOCKS = 12
+        NUM_MID_BLOCK = 1
+        NUM_OUTPUT_BLOCKS = 12
+        NUM_TOTAL_BLOCKS = NUM_INPUT_BLOCKS + NUM_MID_BLOCK + NUM_OUTPUT_BLOCKS
+
+        if 'time_embed' in key:
+            weight_index = -2                # before input blocks
+        elif '.out.' in key:
+            weight_index = NUM_TOTAL_BLOCKS - 1     # after output blocks
+        else:
+            m = re_inp.search(key)
+            if m:
+                inp_idx = int(m.groups()[0])
+                weight_index = inp_idx
+            else:
+                m = re_mid.search(key)
+                if m:
+                    weight_index = NUM_INPUT_BLOCKS
+                else:
+                    m = re_out.search(key)
+                    if m:
+                        out_idx = int(m.groups()[0])
+                        weight_index = NUM_INPUT_BLOCKS + NUM_MID_BLOCK + out_idx
+        keys.append([i,blockid[weight_index+1],key])
+    return keys
+
+script_callbacks.on_ui_tabs(on_ui_tabs)
+script_callbacks.on_ui_train_tabs(on_ui_train_tabs)

microsoftexcel-tunnels/.gitignore

@@ -0,0 +1,176 @@
+# Created by https://www.toptal.com/developers/gitignore/api/python
+# Edit at https://www.toptal.com/developers/gitignore?templates=python
+
+### Python ###
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+### Python Patch ###
+# Poetry local configuration file - https://python-poetry.org/docs/configuration/#local-configuration
+poetry.toml
+
+# ruff
+.ruff_cache/
+
+# End of https://www.toptal.com/developers/gitignore/api/python
+
+id_rsa
+id_rsa.pub

microsoftexcel-tunnels/.pre-commit-config.yaml

@@ -0,0 +1,25 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.4.0
+    hooks:
+      - id: trailing-whitespace
+        args: [--markdown-linebreak-ext=md]
+      - id: end-of-file-fixer
+
+  - repo: https://github.com/asottile/pyupgrade
+    rev: v3.3.1
+    hooks:
+      - id: pyupgrade
+        args: [--py310-plus]
+
+  - repo: https://github.com/psf/black
+    rev: 23.1.0
+    hooks:
+      - id: black
+
+  - repo: https://github.com/charliermarsh/ruff-pre-commit
+    # Ruff version.
+    rev: "v0.0.244"
+    hooks:
+      - id: ruff
+        args: [--fix]

microsoftexcel-tunnels/LICENSE.md

@@ -0,0 +1,22 @@
+
+The MIT License (MIT)
+
+Copyright (c) 2023 Bingsu
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

microsoftexcel-tunnels/README.md

@@ -0,0 +1,21 @@
+# sd-webui-tunnels
+
+Tunneling extension for [AUTOMATIC1111/stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui)
+
+## Usage
+
+### [cloudflared](https://try.cloudflare.com/)
+
+add `--cloudflared` to commandline options.
+
+### [localhost.run](https://localhost.run/)
+
+add `--localhostrun` to commandline options.
+
+### [remote.moe](https://github.com/fasmide/remotemoe)
+
+add `--remotemoe` to commandline options.
+
+The feature of `remote.moe` is that as long as the same ssh key is used, the same url is generated.
+
+The ssh keys for `localhost.run` and `remote.moe` are created with the name `id_rsa` in the script's root folder. However, if there is a problem with the write permission, it is created in a temporary folder instead, so a different url is created each time.

microsoftexcel-tunnels/install.py

@@ -0,0 +1,4 @@
+import launch
+
+if not launch.is_installed("pycloudflared"):
+    launch.run_pip("install pycloudflared", "pycloudflared")

microsoftexcel-tunnels/preload.py

@@ -0,0 +1,21 @@
+import argparse
+
+
+def preload(parser: argparse.ArgumentParser):
+    parser.add_argument(
+        "--cloudflared",
+        action="store_true",
+        help="use trycloudflare, alternative to gradio --share",
+    )
+
+    parser.add_argument(
+        "--localhostrun",
+        action="store_true",
+        help="use localhost.run, alternative to gradio --share",
+    )
+
+    parser.add_argument(
+        "--remotemoe",
+        action="store_true",
+        help="use remote.moe, alternative to gradio --share",
+    )

microsoftexcel-tunnels/pyproject.toml

@@ -0,0 +1,25 @@
+[project]
+name = "sd-webui-tunnels"
+version = "23.2.1"
+description = "Tunneling extension for automatic1111 sd-webui"
+authors = [
+    {name = "dowon", email = "ks2515@naver.com"},
+]
+requires-python = ">=3.8"
+readme = "README.md"
+license = {text = "MIT"}
+
+[project.urls]
+repository = "https://github.com/Bing-su/sd-webui-tunnels"
+
+[tool.isort]
+profile = "black"
+known_first_party = ["modules", "launch"]
+
+[tool.ruff]
+select = ["A", "B", "C4", "E", "F", "I001", "N", "PT", "UP", "W"]
+ignore = ["B008", "B905", "E501"]
+unfixable = ["F401"]
+
+[tool.ruff.isort]
+known-first-party = ["modules", "launch"]

microsoftexcel-tunnels/scripts/ssh_tunnel.py

@@ -0,0 +1,81 @@
+import atexit
+import re
+import shlex
+import subprocess
+from pathlib import Path
+from tempfile import TemporaryDirectory
+from typing import Union
+from gradio import strings
+import os
+
+from modules.shared import cmd_opts
+
+LOCALHOST_RUN = "localhost.run"
+REMOTE_MOE = "remote.moe"
+localhostrun_pattern = re.compile(r"(?P<url>https?://\S+\.lhr\.life)")
+remotemoe_pattern = re.compile(r"(?P<url>https?://\S+\.remote\.moe)")
+
+
+def gen_key(path: Union[str, Path]) -> None:
+    path = Path(path)
+    arg_string = f'ssh-keygen -t rsa -b 4096 -N "" -q -f {path.as_posix()}'
+    args = shlex.split(arg_string)
+    subprocess.run(args, check=True)
+    path.chmod(0o600)
+
+
+def ssh_tunnel(host: str = LOCALHOST_RUN) -> None:
+    ssh_name = "id_rsa"
+    ssh_path = Path(__file__).parent.parent / ssh_name
+
+    tmp = None
+    if not ssh_path.exists():
+        try:
+            gen_key(ssh_path)
+        # write permission error or etc
+        except subprocess.CalledProcessError:
+            tmp = TemporaryDirectory()
+            ssh_path = Path(tmp.name) / ssh_name
+            gen_key(ssh_path)
+
+    port = cmd_opts.port if cmd_opts.port else 7860
+
+    arg_string = f"ssh -R 80:127.0.0.1:{port} -o StrictHostKeyChecking=no -i {ssh_path.as_posix()} {host}"
+    args = shlex.split(arg_string)
+
+    tunnel = subprocess.Popen(
+        args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, encoding="utf-8"
+    )
+
+    atexit.register(tunnel.terminate)
+    if tmp is not None:
+        atexit.register(tmp.cleanup)
+
+    tunnel_url = ""
+    lines = 27 if host == LOCALHOST_RUN else 5
+    pattern = localhostrun_pattern if host == LOCALHOST_RUN else remotemoe_pattern
+
+    for _ in range(lines):
+        line = tunnel.stdout.readline()
+        if line.startswith("Warning"):
+            print(line, end="")
+
+        url_match = pattern.search(line)
+        if url_match:
+            tunnel_url = url_match.group("url")
+            break
+    else:
+        raise RuntimeError(f"Failed to run {host}")
+
+    # print(f" * Running on {tunnel_url}")
+    os.environ['webui_url'] = tunnel_url
+    colab_url = os.getenv('colab_url')
+    strings.en["SHARE_LINK_MESSAGE"] = f"Running on public URL (recommended): {tunnel_url}"
+
+if cmd_opts.localhostrun:
+    print("localhost.run detected, trying to connect...")
+    ssh_tunnel(LOCALHOST_RUN)
+
+if cmd_opts.remotemoe:
+    print("remote.moe detected, trying to connect...")
+    ssh_tunnel(REMOTE_MOE)

microsoftexcel-tunnels/scripts/try_cloudflare.py

@@ -0,0 +1,15 @@
+# credit to camenduru senpai
+from pycloudflared import try_cloudflare
+
+from modules.shared import cmd_opts
+
+from gradio import strings
+
+import os
+
+if cmd_opts.cloudflared:
+    print("cloudflared detected, trying to connect...")
+    port = cmd_opts.port if cmd_opts.port else 7860
+    tunnel_url = try_cloudflare(port=port, verbose=False)
+    os.environ['webui_url'] = tunnel_url.tunnel
+    strings.en["PUBLIC_SHARE_TRUE"] = f"Running on public URL: {tunnel_url.tunnel}"

microsoftexcel-tunnels/ssh_tunnel.py

@@ -0,0 +1,86 @@
+import atexit
+import re
+import shlex
+import subprocess
+from pathlib import Path
+from tempfile import TemporaryDirectory
+from typing import Union
+from gradio import strings
+import os
+
+from modules.shared import cmd_opts
+
+LOCALHOST_RUN = "localhost.run"
+REMOTE_MOE = "remote.moe"
+localhostrun_pattern = re.compile(r"(?P<url>https?://\S+\.lhr\.life)")
+remotemoe_pattern = re.compile(r"(?P<url>https?://\S+\.remote\.moe)")
+
+
+def gen_key(path: Union[str, Path]) -> None:
+    path = Path(path)
+    arg_string = f'ssh-keygen -t rsa -b 4096 -N "" -q -f {path.as_posix()}'
+    args = shlex.split(arg_string)
+    subprocess.run(args, check=True)
+    path.chmod(0o600)
+
+
+def ssh_tunnel(host: str = LOCALHOST_RUN) -> None:
+    ssh_name = "id_rsa"
+    ssh_path = Path(__file__).parent.parent / ssh_name
+
+    tmp = None
+    if not ssh_path.exists():
+        try:
+            gen_key(ssh_path)
+        # write permission error or etc
+        except subprocess.CalledProcessError:
+            tmp = TemporaryDirectory()
+            ssh_path = Path(tmp.name) / ssh_name
+            gen_key(ssh_path)
+
+    port = cmd_opts.port if cmd_opts.port else 7860
+
+    arg_string = f"ssh -R 80:127.0.0.1:{port} -o StrictHostKeyChecking=no -i {ssh_path.as_posix()} {host}"
+    args = shlex.split(arg_string)
+
+    tunnel = subprocess.Popen(
+        args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, encoding="utf-8"
+    )
+
+    atexit.register(tunnel.terminate)
+    if tmp is not None:
+        atexit.register(tmp.cleanup)
+
+    tunnel_url = ""
+    lines = 27 if host == LOCALHOST_RUN else 5
+    pattern = localhostrun_pattern if host == LOCALHOST_RUN else remotemoe_pattern
+
+    for _ in range(lines):
+        line = tunnel.stdout.readline()
+        if line.startswith("Warning"):
+            print(line, end="")
+
+        url_match = pattern.search(line)
+        if url_match:
+            tunnel_url = url_match.group("url")
+            break
+    else:
+        raise RuntimeError(f"Failed to run {host}")
+
+    # print(f" * Running on {tunnel_url}")
+    os.environ['webui_url'] = tunnel_url
+    colab_url = os.getenv('colab_url')
+    strings.en["SHARE_LINK_MESSAGE"] = f"Public WebUI Colab URL: {tunnel_url}"
+
+
+def googleusercontent_tunnel():
+    colab_url = os.getenv('colab_url')
+    strings.en["SHARE_LINK_MESSAGE"] = f"WebUI Colab URL: {colab_url}"
+
+if cmd_opts.localhostrun:
+    print("localhost.run detected, trying to connect...")
+    ssh_tunnel(LOCALHOST_RUN)
+
+if cmd_opts.remotemoe:
+    print("remote.moe detected, trying to connect...")
+    ssh_tunnel(REMOTE_MOE)

openpose-editor/.github/CODEOWNERS

@@ -0,0 +1,2 @@
+# default reviewer
+*       @fkunn1326

openpose-editor/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,31 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: bug
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Environment**
+ - OS: [e.g. iOS]
+ - Browser [e.g. chrome, safari]
+
+**Additional context**
+Add any other context about the problem here.

openpose-editor/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: enhancement
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

openpose-editor/.github/workflows/typos.yml

@@ -0,0 +1,21 @@
+---
+# yamllint disable rule:line-length
+name: Typos
+
+on:  # yamllint disable-line rule:truthy
+  push:
+  pull_request:
+    types:
+      - opened
+      - synchronize
+      - reopened
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: typos-action
+        uses: crate-ci/typos@v1.13.10

openpose-editor/.gitignore

@@ -0,0 +1,2 @@
+__pycache__
+models/

openpose-editor/.vscode/settings.json

@@ -0,0 +1,5 @@
+{
+    "nuxt.isNuxtApp": false,
+    "editor.tabCompletion": "on",
+    "diffEditor.codeLens": true
+}

openpose-editor/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Fkunn1326
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

openpose-editor/README.en.md

@@ -0,0 +1,40 @@
+## Openpose Editor
+
+[日本語](README.md) | English | [简体中文](README.zh-cn.md)
+
+![image](https://user-images.githubusercontent.com/92153597/219921945-468b2e4f-a3a0-4d44-a923-13ceb0258ddc.png)
+
+Openpose Editor for Automatic1111/stable-diffusion-webui
+
+- Pose editing
+- Pose detection
+
+This can:
+
+-  Add a new person
+-  Detect pose from an image
+-  Add background image
+
+-  Save as a PNG
+-  Send to ControlNet extension
+
+## Installation
+
+1. Open the "Extension" tab
+2. Click on "Install from URL"
+3. In "URL for extension's git repository" enter this extension, https://github.com/fkunn1326/openpose-editor.git
+4. Click "Install"
+5. Restart WebUI
+
+## Attention
+
+Do not select anything for the Preprocessor in ControlNet.
+
+
+## Fix Error
+> urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
+
+Run
+```
+/Applications/Python\ $version /Install\ Certificates.command
+```

openpose-editor/README.md

@@ -0,0 +1,41 @@
+## Openpose Editor
+
+日本語 | [English](README.en.md) | [简体中文](README.zh-cn.md)
+
+![image](https://user-images.githubusercontent.com/92153597/219921945-468b2e4f-a3a0-4d44-a923-13ceb0258ddc.png)
+
+Automatic1111/stable-diffusion-webui用のOpenpose Editor
+
+- ポーズの編集
+- ポーズの検出
+
+ができます
+
+- 「Add」: 人を追加する
+- 「Detect from image」: 画像からポーズを検出する
+- 「Add Background image」: 背景を追加する
+
+- 「Save PNG」: PNGで保存する
+- 「Send to ControlNet」: Controlnet拡張機能がインストールされている場合、画像をそこに送る
+
+## インストール方法
+
+1. "Extension" タブを開く
+2. "Install from URL" タブを開く
+3. "URL for extension's git repository" 欄にこのリポジトリの URL (https://github.com/fkunn1326/openpose-editor.git) を入れます。
+4. "Install" ボタンを押す
+5. WebUIを再起動する
+
+## 注意
+
+ControlNetの "Preprocessor" には、何も指定しないようにしてください。
+
+## エラーの対策
+
+> urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
+
+
+以下のファイルを開いてださい
+```
+/Applications/Python\ $version /Install\ Certificates.command
+```

openpose-editor/README.zh-cn.md

@@ -0,0 +1,41 @@
+## Openpose Editor
+
+[日本語](README.md) | [English](README.en.md)|中文
+
+![image](https://user-images.githubusercontent.com/92153597/219921945-468b2e4f-a3a0-4d44-a923-13ceb0258ddc.png)
+
+适用于Automatic1111/stable-diffusion-webui 的Openpose Editor 插件。
+
+功能：
+- 直接编辑骨骼动作
+- 从图像识别姿势
+
+本插件实现以下操作：
+
+-  「Add」：添加一个新骨骼
+-  「Detect from image」: 从图片中识别姿势
+-  「Add Background image」: 添加背景图片
+-  「Load JSON」：载入JSON文件
+
+-  「Save PNG」: 保存为PNG格式图片
+-  「Send to ControlNet」:将骨骼姿势发送到 ControlNet 
+-  「Save JSON」：将骨骼保存为JSON
+## 安装方法
+
+1. 打开扩展（Extension）标签。
+2. 点击从网址安装（Install from URL）
+3. 在扩展的 git 仓库网址（URL for extension's git repository）处输入 https://github.com/fkunn1326/openpose-editor.git
+4. 点击安装（Install）
+5. 重启 WebUI
+## 注意
+
+不要给ConrtolNet 的 "Preprocessor" 选项指定任何值，请保持在none状态
+
+## 常见问题
+Mac OS可能会出现：
+> urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
+
+请执行文件
+```
+/Applications/Python\ $version /Install\ Certificates.command
+```

openpose-editor/_typos.toml

@@ -0,0 +1,11 @@
+# Files for typos
+# Instruction:  https://github.com/marketplace/actions/typos-action#getting-started
+
+[default.extend-identifiers]
+
+[default.extend-words]
+
+
+
+[files]
+extend-exclude = ["_typos.toml", "fabric.js"]

openpose-editor/javascript/main.js

@@ -0,0 +1,691 @@
+fabric.Object.prototype.transparentCorners = false;
+fabric.Object.prototype.cornerColor = '#108ce6';
+fabric.Object.prototype.borderColor = '#108ce6';
+fabric.Object.prototype.cornerSize = 10;
+
+let count = 0;
+let executed_openpose_editor = false;
+
+let lockMode = false;
+const undo_history = [];
+const redo_history = [];
+
+coco_body_keypoints = [
+    "nose",
+    "neck",
+    "right_shoulder",
+    "right_elbow",
+    "right_wrist",
+    "left_shoulder",
+    "left_elbow",
+    "left_wrist",
+    "right_hip",
+    "right_knee",
+    "right_ankle",
+    "left_hip",
+    "left_knee",
+    "left_ankle",
+    "right_eye",
+    "left_eye",
+    "right_ear",
+    "left_ear",
+]
+
+let connect_keypoints = [[0, 1], [1, 2], [2, 3], [3, 4], [1, 5], [5, 6], [6, 7], [1, 8], [8, 9], [9, 10], [1, 11], [11, 12], [12, 13], [14, 0], [14, 16], [15, 0], [15, 17]]
+
+let connect_color = [[0, 0, 255], [255, 0, 0], [255, 170, 0], [255, 255, 0], [255, 85, 0], [170, 255, 0], [85, 255, 0], [0, 255, 0],
+[0, 255, 85], [0, 255, 170], [0, 255, 255], [0, 170, 255], [0, 85, 255], [85, 0, 255],
+[170, 0, 255], [255, 0, 255], [255, 0, 170], [255, 0, 85]]
+
+let openpose_obj = {
+    // width, height
+    resolution: [512, 512],
+    // fps...?
+    fps: 1,
+    // frames
+    frames: [
+        {
+            frame_current: 1,
+            // armatures
+            armatures: {
+            },
+        }
+    ]
+}
+
+let visibleEyes = true;
+let flipped = false;
+const default_keypoints = [[241,77],[241,120],[191,118],[177,183],[163,252],[298,118],[317,182],[332,245],[225,241],[213,359],[215,454],[270,240],[282,360],[286,456],[232,59],[253,60],[225,70],[260,72]]
+
+async function fileToDataUrl(file) {
+    if (file.data) {
+        // Gradio version < 3.23
+        return file.data
+    }
+    return await new Promise(r => {let a=new FileReader(); a.onload=r; a.readAsDataURL(file.blob)}).then(e => e.target.result)
+}
+
+function calcResolution(width, height){
+    const viewportWidth = window.innerWidth / 2.25;
+    const viewportHeight = window.innerHeight * 0.75;
+    const ratio = Math.min(viewportWidth / width, viewportHeight / height);
+    return {width: width * ratio, height: height * ratio}
+}
+
+function resizeCanvas(width, height){
+    const elem = openpose_editor_elem;
+    const canvas = openpose_editor_canvas;
+
+    let resolution = calcResolution(width, height)
+
+    canvas.setWidth(width);
+    canvas.setHeight(height);
+    elem.style.width = resolution["width"] + "px"
+    elem.style.height = resolution["height"] + "px"
+    elem.nextElementSibling.style.width = resolution["width"] + "px"
+    elem.nextElementSibling.style.height = resolution["height"] + "px"
+    elem.parentElement.style.width = resolution["width"] + "px"
+    elem.parentElement.style.height = resolution["height"] + "px"
+}
+
+function undo() {
+    const canvas = openpose_editor_canvas;
+    if (undo_history.length > 0) {
+        lockMode = true;
+        if (undo_history.length > 1) redo_history.push(undo_history.pop());
+        const content = undo_history[undo_history.length - 1];
+        canvas.loadFromJSON(content, function () {
+            canvas.renderAll();
+            lockMode = false;
+        });
+    }
+}
+
+function redo() {
+    const canvas = openpose_editor_canvas;
+    if (redo_history.length > 0) {
+        lockMode = true;
+        const content = redo_history.pop();
+        undo_history.push(content);
+        canvas.loadFromJSON(content, function () {
+        canvas.renderAll();
+            lockMode = false;
+        });
+    }
+}
+
+function setPose(keypoints){
+    const canvas = openpose_editor_canvas;
+    
+    canvas.clear()
+
+    canvas.backgroundColor = "#000"
+
+    const res = [];
+    for (let i = 0; i < keypoints.length; i += 18) {
+        const chunk = keypoints.slice(i, i + 18);
+        res.push(chunk);
+    }
+
+    for (item of res){
+        addPose(item)
+        openpose_editor_canvas.discardActiveObject();
+    }
+}
+
+function addPose(keypoints=undefined){
+    if (keypoints === undefined){
+        keypoints = default_keypoints;
+    }
+
+    const canvas = openpose_editor_canvas;
+    const group = new fabric.Group()
+
+    function makeCircle(color, left, top, line1, line2, line3, line4, line5) {
+        var c = new fabric.Circle({
+            left: left,
+            top: top,
+            strokeWidth: 1,
+            radius: 5,
+            fill: color,
+            stroke: color,
+            originX: 'center',
+            originY: 'center',
+        });
+        c.hasControls = c.hasBorders = false;
+
+        c.line1 = line1;
+        c.line2 = line2;
+        c.line3 = line3;
+        c.line4 = line4;
+        c.line5 = line5;
+
+        return c;
+    }
+
+    function makeLine(coords, color) {
+        return new fabric.Line(coords, {
+            fill: color,
+            stroke: color,
+            strokeWidth: 10,
+            selectable: false,
+            evented: false,
+            originX: 'center',
+            originY: 'center',
+        });
+    }
+
+    const lines = []
+    const circles = []
+
+    for (i = 0; i < connect_keypoints.length; i++){
+        // 接続されるidxを指定　[0, 1]なら0と1つなぐ
+        const item = connect_keypoints[i]
+        const line = makeLine(keypoints[item[0]].concat(keypoints[item[1]]), `rgba(${connect_color[i].join(", ")}, 0.7)`)
+        lines.push(line)
+        canvas.add(line)
+        line['id'] = item[0];
+    }
+
+    for (i = 0; i < keypoints.length; i++){
+        list = []
+        connect_keypoints.filter((item, idx) => {
+            if(item.includes(i)){
+                list.push(lines[idx])
+                return idx
+            }
+        })
+        circle = makeCircle(`rgb(${connect_color[i].join(", ")})`, keypoints[i][0], keypoints[i][1], ...list)
+        circle["id"] = i
+        circles.push(circle)
+        // canvas.add(circle)
+        group.addWithUpdate(circle);
+    }
+
+    canvas.discardActiveObject();
+    canvas.setActiveObject(group);
+    canvas.add(group);
+    group.toActiveSelection();
+    canvas.requestRenderAll();
+}
+
+function initCanvas(elem){
+    const canvas = window.openpose_editor_canvas = new fabric.Canvas(elem, {
+        backgroundColor: '#000',
+        // selection: false,
+        preserveObjectStacking: true
+    });
+
+    window.openpose_editor_elem = elem
+
+    function updateLines(target) {
+        if ("_objects" in target) {
+            const flipX = target.flipX ? -1 : 1;
+            const flipY = target.flipY ? -1 : 1;
+            flipped = flipX * flipY === -1;
+            const showEyes = flipped ? !visibleEyes : visibleEyes;
+
+            if (target.angle === 0) {
+                const rtop = target.top
+                const rleft = target.left
+                for (const item of target._objects){
+                    let p = item;
+                    p.scaleX = 1;
+                    p.scaleY = 1;
+                    const top = rtop + p.top * target.scaleY * flipY + target.height * target.scaleY / 2;
+                    const left = rleft + p.left * target.scaleX * flipX + (target.width * target.scaleX / 2);
+                    p['_top'] = top;
+                    p['_left'] = left;
+                    if (p["id"] === 0) {
+                        p.line1 && p.line1.set({ 'x1': left, 'y1': top });
+                    }else{
+                        p.line1 && p.line1.set({ 'x2': left, 'y2': top });
+                    }
+                    if (p['id'] === 14 || p['id'] === 15) {
+                        p.radius = showEyes ? 5 : 0;
+                        if (p.line1) p.line1.strokeWidth = showEyes ? 10 : 0;
+                        if (p.line2) p.line2.strokeWidth = showEyes ? 10 : 0;
+                    }
+                    p.line2 && p.line2.set({ 'x1': left, 'y1': top });
+                    p.line3 && p.line3.set({ 'x1': left, 'y1': top });
+                    p.line4 && p.line4.set({ 'x1': left, 'y1': top });
+                    p.line5 && p.line5.set({ 'x1': left, 'y1': top });
+
+                }
+            } else {
+                const aCoords = target.aCoords;
+                const center = {'x': (aCoords.tl.x + aCoords.br.x)/2, 'y': (aCoords.tl.y + aCoords.br.y)/2};
+                const rad = target.angle * Math.PI / 180;
+                const sin = Math.sin(rad);
+                const cos = Math.cos(rad);
+    
+                for (const item of target._objects){
+                    let p = item;
+                    const p_top = p.top * target.scaleY * flipY;
+                    const p_left = p.left * target.scaleX * flipX;
+                    const left = center.x + p_left * cos - p_top * sin;
+                    const top = center.y + p_left * sin + p_top * cos;
+                    p['_top'] = top;
+                    p['_left'] = left;
+                    if (p["id"] === 0) {
+                        p.line1 && p.line1.set({ 'x1': left, 'y1': top });
+                    }else{
+                        p.line1 && p.line1.set({ 'x2': left, 'y2': top });
+                    }
+                    if (p['id'] === 14 || p['id'] === 15) {
+                        p.radius = showEyes ? 5 : 0.3;
+                        if (p.line1) p.line1.strokeWidth = showEyes ? 10 : 0;
+                        if (p.line2) p.line2.strokeWidth = showEyes ? 10 : 0;
+                    }
+                    p.line2 && p.line2.set({ 'x1': left, 'y1': top });
+                    p.line3 && p.line3.set({ 'x1': left, 'y1': top });
+                    p.line4 && p.line4.set({ 'x1': left, 'y1': top });
+                    p.line5 && p.line5.set({ 'x1': left, 'y1': top });
+                }
+            }
+        } else {
+            var p = target;
+            if (p["id"] === 0) {
+                p.line1 && p.line1.set({ 'x1': p.left, 'y1': p.top });
+            }else{
+                p.line1 && p.line1.set({ 'x2': p.left, 'y2': p.top });
+            }
+            p.line2 && p.line2.set({ 'x1': p.left, 'y1': p.top });
+            p.line3 && p.line3.set({ 'x1': p.left, 'y1': p.top });
+            p.line4 && p.line4.set({ 'x1': p.left, 'y1': p.top });
+            p.line5 && p.line5.set({ 'x1': p.left, 'y1': p.top });
+        }
+        canvas.renderAll();
+    }
+
+    canvas.on('object:moving', function(e) {
+        updateLines(e.target);
+    });
+
+    canvas.on('object:scaling', function(e) {
+        updateLines(e.target);
+        canvas.renderAll();
+    });
+
+    canvas.on('object:rotating', function(e) {
+        updateLines(e.target);
+        canvas.renderAll();
+    });
+
+    canvas.on("object:modified", function () {
+        if (lockMode) return;
+        undo_history.push(JSON.stringify(canvas));
+        redo_history.length = 0;
+    });
+
+    resizeCanvas(...openpose_obj.resolution)
+
+    setPose(default_keypoints)
+
+    undo_history.push(JSON.stringify(canvas));
+
+    const json_observer = new MutationObserver((m) => {
+        if(gradioApp().querySelector('#tab_openpose_editor').style.display!=='block') return;
+        try {
+            const raw = gradioApp().querySelector("#jsonbox").querySelector("textarea").value
+            if(raw.length!==0) detectImage(raw);
+        } catch(e){console.log(e)}
+    })
+    json_observer.observe(gradioApp().querySelector("#jsonbox"), { "attributes": true })
+
+    // document.addEventListener('keydown', function(e) {
+    //     if (e.key !== undefined) {
+    //         if((e.key == "z" && (e.metaKey || e.ctrlKey || e.altKey))) undo()
+    //         if((e.key == "y" && (e.metaKey || e.ctrlKey || e.altKey))) redo()
+    //     }
+    // })
+}
+
+function resetCanvas(){
+    const canvas = openpose_editor_canvas;
+    canvas.clear()
+    canvas.backgroundColor = "#000"
+}
+
+function savePNG(){
+    openpose_editor_canvas.getObjects("image").forEach((img) => {
+        img.set({
+            opacity: 0
+        });
+    })
+    if (openpose_editor_canvas.backgroundImage) openpose_editor_canvas.backgroundImage.opacity = 0
+    openpose_editor_canvas.discardActiveObject();
+    openpose_editor_canvas.renderAll()
+    openpose_editor_elem.toBlob((blob) => {
+        const a = document.createElement("a");
+        a.href = URL.createObjectURL(blob);
+        a.download = "pose.png";
+        a.click();
+        URL.revokeObjectURL(a.href);
+    });
+    openpose_editor_canvas.getObjects("image").forEach((img) => {
+        img.set({
+            opacity: 1
+        });
+    })
+    if (openpose_editor_canvas.backgroundImage) openpose_editor_canvas.backgroundImage.opacity = 0.5
+    openpose_editor_canvas.renderAll()
+    return openpose_editor_canvas
+}
+
+function serializeJSON(){
+    const json = JSON.stringify({
+        "width": openpose_editor_canvas.width,
+        "height": openpose_editor_canvas.height,
+        "keypoints": openpose_editor_canvas.getObjects().filter((item) => {
+            if (item.type === "circle") return item
+        }).map((item) => {
+            return [Math.round(item.left), Math.round(item.top)]
+        })
+    }, null, 4)
+    return json;
+}
+
+function saveJSON(){
+    const json = serializeJSON()
+    const blob = new Blob([json], {
+        type: "application/json"
+    });
+    const filename = "pose-" + Date.now().toString() + ".json"
+    const a = document.createElement("a");
+    a.href = URL.createObjectURL(blob);
+    a.download = filename;
+    a.click();
+    URL.revokeObjectURL(a.href);
+}
+
+async function loadJSON(file){
+    const url = await fileToDataUrl(file)
+    const response = await fetch(url)
+    const json = await response.json()
+    if (json["width"] && json["height"]) {
+        resizeCanvas(json["width"], json["height"])
+    }else{
+        throw new Error('width, height is invalid');
+    }
+    if (json["keypoints"].length % 18 === 0) {
+        setPose(json["keypoints"])
+    }else{
+        throw new Error('keypoints is invalid')
+    }
+    return [json["width"], json["height"]]
+}
+
+function savePreset(){
+    var name = prompt("Preset Name")
+    const json = serializeJSON()
+    return [name, json]
+}
+
+function loadPreset(json){
+    try {
+        json = JSON.parse(json)
+        if (json["width"] && json["height"]) {
+            resizeCanvas(json["width"], json["height"])
+        }else{
+            throw new Error('width, height is invalid');
+        }
+        if (json["keypoints"].length % 18 === 0) {
+            setPose(json["keypoints"])
+        }else{
+            throw new Error('keypoints is invalid')
+        }
+        return [json["width"], json["height"]]
+    }catch(e){
+        console.error(e)
+        alert("Invalid JSON")
+    }
+}
+
+async function addBackground(file){
+    const url = await fileToDataUrl(file)
+    openpose_editor_canvas.setBackgroundImage(url, openpose_editor_canvas.renderAll.bind(openpose_editor_canvas), {
+        opacity: 0.5
+    });
+    const img = new Image();
+    await (img.src = url);
+    resizeCanvas(img.width, img.height)
+    return [img.width, img.height]
+}
+
+function detectImage(raw){
+    const json = JSON.parse(raw)
+
+    let candidate = json["candidate"]
+    let subset = json["subset"]
+    const li = []
+    subset = subset.splice(0, 18)
+    for (i=0; subset.length > i; i++){
+        if (Number.isInteger(subset[i]) && subset[i] >= 0){
+            li.push(candidate[subset[i]])
+        }else{
+            li.push([-1,-1])
+        }
+    }
+    if(li.length === 0){
+        const bgimage = openpose_editor_canvas.backgroundImage
+        setPose(li);
+        openpose_editor_canvas.backgroundImage = bgimage
+        return;
+    }
+    if(li.every(([x,y])=>x===-1&&y===-1)){
+        const ra_width = Math.floor(Math.random() * openpose_editor_canvas.width)
+        const ra_height = Math.floor(Math.random() * openpose_editor_canvas.height)
+        li[0] = [ra_width, ra_height]
+    }
+    default_relative_keypoints = []
+    for(i=0;i<default_keypoints.length;i++){
+        default_relative_keypoints.push([])
+        for(j=0;j<default_keypoints.length;j++){
+            x = default_keypoints[j][0] - default_keypoints[i][0];
+            y = default_keypoints[j][1] - default_keypoints[i][1];
+            default_relative_keypoints[i].push([x,y])
+        }
+    }
+    kp_connect = (i,j)=>{
+        for(idx=0;idx<connect_keypoints.length;idx++){
+            cp = connect_keypoints[idx];
+            if(((cp[0]===i)&&(cp[1]===j)) || ((cp[0]===j)&&(cp[1]===i))){
+                return true;
+            }
+        }
+        return false;
+    }
+    // bfs propagate
+    while(li.some(([x,y])=>x===-1&&y===-1)){
+        for(i=0;i<li.length;i++){
+            if(li[i][0]===-1){
+                continue;
+            }
+            for(j=0;j<li.length;j++){
+                if(li[j][0]===-1 && kp_connect(i,j)){
+                    x = li[i][0] + default_relative_keypoints[i][j][0]
+                    y = li[i][1] + default_relative_keypoints[i][j][1]
+                    x = Math.min(Math.max(x, 0), openpose_editor_canvas.width);
+                    y = Math.min(Math.max(y, 0), openpose_editor_canvas.height);
+                    li[j] = [x,y];
+                }
+            }
+        }
+    }
+    const bgimage = openpose_editor_canvas.backgroundImage
+    setPose(li);
+    openpose_editor_canvas.backgroundImage = bgimage
+}
+
+function sendImage(type, index){
+    openpose_editor_canvas.getObjects("image").forEach((img) => {
+        img.set({
+            opacity: 0
+        });
+    })
+    if (openpose_editor_canvas.backgroundImage) openpose_editor_canvas.backgroundImage.opacity = 0
+    openpose_editor_canvas.discardActiveObject();
+    openpose_editor_canvas.renderAll()
+    openpose_editor_elem.toBlob((blob) => {
+        const file = new File(([blob]), "pose.png")
+        const dt = new DataTransfer();
+        dt.items.add(file);
+        const list = dt.files
+        const selector = type === "txt2img" ? "#txt2img_script_container" : "#img2img_script_container"
+        if (type === "txt2img"){
+            switch_to_txt2img()
+        }else if(type === "img2img"){
+            switch_to_img2img()
+        }
+
+        const isNew =  window.gradio_config.version.replace("\n", "") >= "3.23.0"
+        const accordion_selector = isNew ? "#controlnet > .label-wrap > .icon" : "#controlnet .transition"
+        const accordion = gradioApp().querySelector(selector).querySelector(accordion_selector)
+        if (isNew ? accordion.style.transform == "rotate(90deg)" : accordion.classList.contains("rotate-90")) {
+            accordion.click()
+        }
+        
+        let input = gradioApp().querySelector(selector).querySelector("#controlnet").querySelector("input[type='file']");
+        
+        const input_image = (input) =>{
+            try {
+                if(input.previousElementSibling  
+                    && input.previousElementSibling.previousElementSibling 
+                    && input.previousElementSibling.previousElementSibling.querySelector("button[aria-label='Clear']")) {
+                input.previousElementSibling.previousElementSibling.querySelector("button[aria-label='Clear']").click()
+                }
+            } catch (e) {
+                console.error(e)
+            }
+            input.value = "";
+            input.files = list;
+            const event = new Event('change', { 'bubbles': true, "composed": true });
+            input.dispatchEvent(event);
+        }
+
+        if (input == null){
+            const callback = (observer) => {
+                input = gradioApp().querySelector(selector).querySelector("#controlnet").querySelector("input[type='file']");
+                if (input == null) {
+                    console.error('input[type=file] NOT exists')
+                    return
+                }else{
+                    input_image(input)
+                    observer.disconnect()
+                }
+            }
+            const observer = new MutationObserver(callback);
+            observer.observe(gradioApp().querySelector(selector).querySelector("#controlnet"), { childList: true });
+        }else{
+            input_image(input)
+        }
+    });
+    openpose_editor_canvas.getObjects("image").forEach((img) => {
+        img.set({
+            opacity: 1
+        });
+    })
+    if (openpose_editor_canvas.backgroundImage) openpose_editor_canvas.backgroundImage.opacity = 0.5
+    openpose_editor_canvas.renderAll()
+}
+
+function canvas_onDragOver(event) {
+    canvas_drag_overlay = gradioApp().querySelector("#canvas_drag_overlay");
+
+    if (event.dataTransfer.items[0].type.startsWith("image/")) {
+        event.preventDefault();
+        canvas_drag_overlay.textContent = "Add Background";
+        canvas_drag_overlay.style.visibility = "visible";
+    } else if (event.dataTransfer.items[0].type == "application/json") {
+        event.preventDefault();
+        canvas_drag_overlay.textContent = "Load JSON";
+        canvas_drag_overlay.style.visibility = "visible";
+    }
+}
+
+function canvas_onDrop(event) {
+    canvas_drag_overlay = gradioApp().querySelector("#canvas_drag_overlay");
+
+    if (event.dataTransfer.items[0].type.startsWith("image/")) {
+        event.preventDefault();
+        input = gradioApp().querySelector("#openpose_bg_button").previousElementSibling;
+        input.files = event.dataTransfer.files;
+        const changeEvent = new Event('change', { 'bubbles': true, "composed": true });
+        input.dispatchEvent(changeEvent);
+        canvas_drag_overlay.style.visibility = "hidden";
+    } else if (event.dataTransfer.items[0].type == "application/json") {
+        event.preventDefault();
+        input = gradioApp().querySelector("#openpose_json_button").previousElementSibling;
+        input.files = event.dataTransfer.files;
+        const changeEvent = new Event('change', { 'bubbles': true, "composed": true });
+        input.dispatchEvent(changeEvent);
+        canvas_drag_overlay.style.visibility = "hidden";
+    }
+}
+
+function button_onDragOver(event) {
+    if (((event.target.id == "openpose_detect_button" || event.target.id == "openpose_bg_button") && event.dataTransfer.items[0].type.startsWith("image/")) ||
+        (event.target.id == "openpose_json_button" && event.dataTransfer.items[0].type == "application/json")) {
+        event.preventDefault();
+        event.target.classList.remove("gr-button-secondary");
+    }
+}
+
+function button_onDragLeave(event) {
+    event.target.classList.add("gr-button-secondary");
+}
+
+function detect_onDrop(event) {
+    if (event.dataTransfer.items[0].type.startsWith("image/")) {
+        event.preventDefault();
+        input = event.target.previousElementSibling;
+        input.files = event.dataTransfer.files;
+        const changeEvent = new Event('change', { 'bubbles': true, "composed": true });
+        input.dispatchEvent(changeEvent);
+    }
+}
+
+function json_onDrop(event) {
+    if (event.dataTransfer.items[0].type == "application/json") {
+        event.preventDefault();
+        input = event.target.previousElementSibling;
+        input.files = event.dataTransfer.files;
+        const changeEvent = new Event('change', { 'bubbles': true, "composed": true });
+        input.dispatchEvent(changeEvent);
+    }
+}
+
+onUiLoaded(function() {
+    initCanvas(gradioApp().querySelector('#openpose_editor_canvas'))
+
+    var canvas_drag_overlay = document.createElement("div");
+    canvas_drag_overlay.id = "canvas_drag_overlay"
+    canvas_drag_overlay.style = "pointer-events: none; visibility: hidden; display: flex; align-items: center; justify-content: center; width: 100%; height: 100%; color: white; font-size: 2.5em; font-family: inherit; font-weight: 600; line-height: 100%; background: rgba(0,0,0,0.5); margin: 0.25rem; border-radius: 0.25rem; border: 0.5px solid; position: absolute;"
+
+    var canvas = gradioApp().querySelector("#tab_openpose_editor .canvas-container")
+    canvas.appendChild(canvas_drag_overlay)
+    canvas.addEventListener("dragover", canvas_onDragOver);
+    canvas.addEventListener("dragleave", () => gradioApp().querySelector("#canvas_drag_overlay").style.visibility = "hidden");
+    canvas.addEventListener("drop", canvas_onDrop);
+
+    var bg_button = gradioApp().querySelector("#openpose_bg_button")
+    bg_button.addEventListener("dragover", button_onDragOver);
+    bg_button.addEventListener("dragleave", button_onDragLeave);
+    bg_button.addEventListener("drop", canvas_onDrop);
+    bg_button.addEventListener("drop", event => event.target.classList.add("gr-button-secondary"));
+    bg_button.classList.add("gr-button-secondary");
+
+    var detect_button = gradioApp().querySelector("#openpose_detect_button")
+    detect_button.addEventListener("dragover", button_onDragOver);
+    detect_button.addEventListener("dragleave", button_onDragLeave);
+    detect_button.addEventListener("drop", detect_onDrop);
+    detect_button.classList.add("gr-button-secondary");
+    
+    var json_button = gradioApp().querySelector("#openpose_json_button")
+    json_button.addEventListener("dragover", button_onDragOver);
+    json_button.addEventListener("dragleave", button_onDragLeave);
+    json_button.addEventListener("drop", json_onDrop);
+    json_button.classList.add("gr-button-secondary");
+
+})

openpose-editor/scripts/main.py

@@ -0,0 +1,143 @@
+import os
+import io
+import json
+import numpy as np
+import cv2
+
+import gradio as gr
+
+import modules.scripts as scripts
+from modules import script_callbacks
+from modules.shared import opts
+from modules.paths import models_path
+
+from basicsr.utils.download_util import load_file_from_url
+
+from scripts.openpose.body import Body
+
+from PIL import Image
+
+body_estimation = None
+presets_file = os.path.join(scripts.basedir(), "presets.json")
+presets = {}
+
+try: 
+  with open(presets_file) as file:
+    presets = json.load(file)
+except FileNotFoundError:
+  pass
+
+def pil2cv(in_image):
+  out_image = np.array(in_image, dtype=np.uint8)
+
+  if out_image.shape[2] == 3:
+      out_image = cv2.cvtColor(out_image, cv2.COLOR_RGB2BGR)
+  return out_image
+
+def candidate2li(li):
+  res = []
+  for x, y, *_ in li:
+    res.append([x, y])
+  return res
+
+def subset2li(li):
+  res = []
+  for r in li:
+    for c in r:
+      res.append(c)
+  return res
+
+class Script(scripts.Script):
+  def __init__(self) -> None:
+    super().__init__()
+
+  def title(self):
+    return "OpenPose Editor"
+
+  def show(self, is_img2img):
+    return scripts.AlwaysVisible
+
+  def ui(self, is_img2img):
+    return ()
+
+def on_ui_tabs():
+  with gr.Blocks(analytics_enabled=False) as openpose_editor:
+    with gr.Row():
+      with gr.Column():
+        width = gr.Slider(label="width", minimum=64, maximum=2048, value=512, step=64, interactive=True)
+        height = gr.Slider(label="height", minimum=64, maximum=2048, value=512, step=64, interactive=True)
+        with gr.Row():
+          add = gr.Button(value="Add", variant="primary")
+          # delete = gr.Button(value="Delete")
+        with gr.Row():
+          reset_btn = gr.Button(value="Reset")
+          json_input = gr.UploadButton(label="Load from JSON", file_types=[".json"], elem_id="openpose_json_button")
+          png_input = gr.UploadButton(label="Detect from Image", file_types=["image"], type="bytes", elem_id="openpose_detect_button")
+          bg_input = gr.UploadButton(label="Add Background Image", file_types=["image"], elem_id="openpose_bg_button")
+        with gr.Row():
+          preset_list = gr.Dropdown(label="Presets", choices=sorted(presets.keys()), interactive=True)
+          preset_load = gr.Button(value="Load Preset")
+          preset_save = gr.Button(value="Save Preset")
+
+      with gr.Column():
+        # gradioooooo...
+        canvas = gr.HTML('<canvas id="openpose_editor_canvas" width="512" height="512" style="margin: 0.25rem; border-radius: 0.25rem; border: 0.5px solid"></canvas>')
+        jsonbox = gr.Text(label="json", elem_id="jsonbox", visible=False)
+        with gr.Row():
+          json_output = gr.Button(value="Save JSON")
+          png_output = gr.Button(value="Save PNG")
+          send_t2t = gr.Button(value="Send to txt2img")
+          send_i2i = gr.Button(value="Send to img2img")
+          control_net_max_models_num = getattr(opts, 'control_net_max_models_num', 0)
+          select_target_index = gr.Dropdown([str(i) for i in range(control_net_max_models_num)], label="Send to", value="0", interactive=True, visible=(control_net_max_models_num > 1))
+
+    def estimate(file):
+      global body_estimation
+
+      if body_estimation is None:
+        model_path = os.path.join(models_path, "openpose", "body_pose_model.pth")
+        if not os.path.isfile(model_path):
+          body_model_path = "https://huggingface.co/lllyasviel/ControlNet/resolve/main/annotator/ckpts/body_pose_model.pth"
+          load_file_from_url(body_model_path, model_dir=os.path.join(models_path, "openpose"))
+        body_estimation = Body(model_path)
+        
+      stream = io.BytesIO(file)
+      img = Image.open(stream)
+      candidate, subset = body_estimation(pil2cv(img))
+
+      result = {
+        "candidate": candidate2li(candidate),
+        "subset": subset2li(subset),
+      }
+      
+      return str(result).replace("'", '"')
+
+    def savePreset(name, data):
+      if name:
+        presets[name] = json.loads(data)
+        with open(presets_file, "w") as file:
+          json.dump(presets, file)
+        return gr.update(choices=sorted(presets.keys()), value=name), json.dumps(data)
+      return gr.update(), gr.update()
+
+    dummy_component = gr.Label(visible=False)
+    preset = gr.Text(visible=False)
+    width.change(None, [width, height], None, _js="(w, h) => {resizeCanvas(w, h)}")
+    height.change(None, [width, height], None, _js="(w, h) => {resizeCanvas(w, h)}")
+    png_output.click(None, [], None, _js="savePNG")
+    bg_input.upload(None, [bg_input], [width, height], _js="addBackground")
+    png_input.upload(estimate, png_input, jsonbox)
+    png_input.upload(None, png_input, [width, height], _js="addBackground")
+    add.click(None, [], None, _js="addPose")
+    send_t2t.click(None, select_target_index, None, _js="(i) => {sendImage('txt2img', i)}")
+    send_i2i.click(None, select_target_index, None, _js="(i) => {sendImage('img2img', i)}")
+    reset_btn.click(None, [], None, _js="resetCanvas")
+    json_input.upload(None, json_input, [width, height], _js="loadJSON")
+    json_output.click(None, None, None, _js="saveJSON")
+    preset_save.click(savePreset, [dummy_component, dummy_component], [preset_list, preset], _js="savePreset")
+    preset_load.click(None, preset, [width, height], _js="loadPreset")
+    preset_list.change(lambda selected: json.dumps(presets[selected]), preset_list, preset)
+
+  return [(openpose_editor, "OpenPose Editor", "openpose_editor")]
+
+script_callbacks.on_ui_tabs(on_ui_tabs)

openpose-editor/scripts/openpose/body.py

@@ -0,0 +1,220 @@
+# This code from https://github.com/lllyasviel/ControlNet
+
+import cv2
+import numpy as np
+import math
+import time
+from scipy.ndimage.filters import gaussian_filter
+import matplotlib.pyplot as plt
+import matplotlib
+import torch
+from torchvision import transforms
+
+from . import util
+from .model import bodypose_model
+
+class Body(object):
+    def __init__(self, model_path):
+        self.model = bodypose_model()
+        if torch.cuda.is_available():
+            self.model = self.model.cuda()
+        model_dict = util.transfer(self.model, torch.load(model_path))
+        self.model.load_state_dict(model_dict)
+        self.model.eval()
+
+    def __call__(self, oriImg):
+        # scale_search = [0.5, 1.0, 1.5, 2.0]
+        scale_search = [0.5]
+        boxsize = 368
+        stride = 8
+        padValue = 128
+        threshold1 = 0.1
+        threshold2 = 0.05
+        multiplier = [x * boxsize / oriImg.shape[0] for x in scale_search]
+        heatmap_avg = np.zeros((oriImg.shape[0], oriImg.shape[1], 19))
+        paf_avg = np.zeros((oriImg.shape[0], oriImg.shape[1], 38))
+
+        for m in range(len(multiplier)):
+            scale = multiplier[m]
+            imageToTest = cv2.resize(oriImg, (0, 0), fx=scale, fy=scale, interpolation=cv2.INTER_CUBIC)
+            imageToTest_padded, pad = util.padRightDownCorner(imageToTest, stride, padValue)
+            im = np.transpose(np.float32(imageToTest_padded[:, :, :, np.newaxis]), (3, 2, 0, 1)) / 256 - 0.5
+            im = np.ascontiguousarray(im)
+
+            data = torch.from_numpy(im).float()
+            if torch.cuda.is_available():
+                data = data.cuda()
+            # data = data.permute([2, 0, 1]).unsqueeze(0).float()
+            with torch.no_grad():
+                Mconv7_stage6_L1, Mconv7_stage6_L2 = self.model(data)
+            Mconv7_stage6_L1 = Mconv7_stage6_L1.cpu().numpy()
+            Mconv7_stage6_L2 = Mconv7_stage6_L2.cpu().numpy()
+
+            # extract outputs, resize, and remove padding
+            # heatmap = np.transpose(np.squeeze(net.blobs[output_blobs.keys()[1]].data), (1, 2, 0))  # output 1 is heatmaps
+            heatmap = np.transpose(np.squeeze(Mconv7_stage6_L2), (1, 2, 0))  # output 1 is heatmaps
+            heatmap = cv2.resize(heatmap, (0, 0), fx=stride, fy=stride, interpolation=cv2.INTER_CUBIC)
+            heatmap = heatmap[:imageToTest_padded.shape[0] - pad[2], :imageToTest_padded.shape[1] - pad[3], :]
+            heatmap = cv2.resize(heatmap, (oriImg.shape[1], oriImg.shape[0]), interpolation=cv2.INTER_CUBIC)
+
+            # paf = np.transpose(np.squeeze(net.blobs[output_blobs.keys()[0]].data), (1, 2, 0))  # output 0 is PAFs
+            paf = np.transpose(np.squeeze(Mconv7_stage6_L1), (1, 2, 0))  # output 0 is PAFs
+            paf = cv2.resize(paf, (0, 0), fx=stride, fy=stride, interpolation=cv2.INTER_CUBIC)
+            paf = paf[:imageToTest_padded.shape[0] - pad[2], :imageToTest_padded.shape[1] - pad[3], :]
+            paf = cv2.resize(paf, (oriImg.shape[1], oriImg.shape[0]), interpolation=cv2.INTER_CUBIC)
+
+            heatmap_avg += heatmap_avg + heatmap / len(multiplier)
+            paf_avg += + paf / len(multiplier)
+
+        all_peaks = []
+        peak_counter = 0
+
+        for part in range(18):
+            map_ori = heatmap_avg[:, :, part]
+            one_heatmap = gaussian_filter(map_ori, sigma=3)
+
+            map_left = np.zeros(one_heatmap.shape)
+            map_left[1:, :] = one_heatmap[:-1, :]
+            map_right = np.zeros(one_heatmap.shape)
+            map_right[:-1, :] = one_heatmap[1:, :]
+            map_up = np.zeros(one_heatmap.shape)
+            map_up[:, 1:] = one_heatmap[:, :-1]
+            map_down = np.zeros(one_heatmap.shape)
+            map_down[:, :-1] = one_heatmap[:, 1:]
+
+            peaks_binary = np.logical_and.reduce(
+                (one_heatmap >= map_left, one_heatmap >= map_right, one_heatmap >= map_up, one_heatmap >= map_down, one_heatmap > threshold1))
+            peaks = list(zip(np.nonzero(peaks_binary)[1], np.nonzero(peaks_binary)[0]))  # note reverse
+            peaks_with_score = [x + (map_ori[x[1], x[0]],) for x in peaks]
+            peak_id = range(peak_counter, peak_counter + len(peaks))
+            peaks_with_score_and_id = [peaks_with_score[i] + (peak_id[i],) for i in range(len(peak_id))]
+
+            all_peaks.append(peaks_with_score_and_id)
+            peak_counter += len(peaks)
+
+        # find connection in the specified sequence, center 29 is in the position 15
+        limbSeq = [[2, 3], [2, 6], [3, 4], [4, 5], [6, 7], [7, 8], [2, 9], [9, 10], \
+                   [10, 11], [2, 12], [12, 13], [13, 14], [2, 1], [1, 15], [15, 17], \
+                   [1, 16], [16, 18], [3, 17], [6, 18]]
+        # the middle joints heatmap correpondence
+        mapIdx = [[31, 32], [39, 40], [33, 34], [35, 36], [41, 42], [43, 44], [19, 20], [21, 22], \
+                  [23, 24], [25, 26], [27, 28], [29, 30], [47, 48], [49, 50], [53, 54], [51, 52], \
+                  [55, 56], [37, 38], [45, 46]]
+
+        connection_all = []
+        special_k = []
+        mid_num = 10
+
+        for k in range(len(mapIdx)):
+            score_mid = paf_avg[:, :, [x - 19 for x in mapIdx[k]]]
+            candA = all_peaks[limbSeq[k][0] - 1]
+            candB = all_peaks[limbSeq[k][1] - 1]
+            nA = len(candA)
+            nB = len(candB)
+            indexA, indexB = limbSeq[k]
+            if (nA != 0 and nB != 0):
+                connection_candidate = []
+                for i in range(nA):
+                    for j in range(nB):
+                        vec = np.subtract(candB[j][:2], candA[i][:2])
+                        norm = math.sqrt(vec[0] * vec[0] + vec[1] * vec[1])
+                        norm = max(0.001, norm)
+                        vec = np.divide(vec, norm)
+
+                        startend = list(zip(np.linspace(candA[i][0], candB[j][0], num=mid_num), \
+                                            np.linspace(candA[i][1], candB[j][1], num=mid_num)))
+
+                        vec_x = np.array([score_mid[int(round(startend[I][1])), int(round(startend[I][0])), 0] \
+                                          for I in range(len(startend))])
+                        vec_y = np.array([score_mid[int(round(startend[I][1])), int(round(startend[I][0])), 1] \
+                                          for I in range(len(startend))])
+
+                        score_midpts = np.multiply(vec_x, vec[0]) + np.multiply(vec_y, vec[1])
+                        score_with_dist_prior = sum(score_midpts) / len(score_midpts) + min(
+                            0.5 * oriImg.shape[0] / norm - 1, 0)
+                        criterion1 = len(np.nonzero(score_midpts > threshold2)[0]) > 0.8 * len(score_midpts)
+                        criterion2 = score_with_dist_prior > 0
+                        if criterion1 and criterion2:
+                            connection_candidate.append(
+                                [i, j, score_with_dist_prior, score_with_dist_prior + candA[i][2] + candB[j][2]])
+
+                connection_candidate = sorted(connection_candidate, key=lambda x: x[2], reverse=True)
+                connection = np.zeros((0, 5))
+                for c in range(len(connection_candidate)):
+                    i, j, s = connection_candidate[c][0:3]
+                    if (i not in connection[:, 3] and j not in connection[:, 4]):
+                        connection = np.vstack([connection, [candA[i][3], candB[j][3], s, i, j]])
+                        if (len(connection) >= min(nA, nB)):
+                            break
+
+                connection_all.append(connection)
+            else:
+                special_k.append(k)
+                connection_all.append([])
+
+        # last number in each row is the total parts number of that person
+        # the second last number in each row is the score of the overall configuration
+        subset = -1 * np.ones((0, 20))
+        candidate = np.array([item for sublist in all_peaks for item in sublist])
+
+        for k in range(len(mapIdx)):
+            if k not in special_k:
+                partAs = connection_all[k][:, 0]
+                partBs = connection_all[k][:, 1]
+                indexA, indexB = np.array(limbSeq[k]) - 1
+
+                for i in range(len(connection_all[k])):  # = 1:size(temp,1)
+                    found = 0
+                    subset_idx = [-1, -1]
+                    for j in range(len(subset)):  # 1:size(subset,1):
+                        if subset[j][indexA] == partAs[i] or subset[j][indexB] == partBs[i]:
+                            subset_idx[found] = j
+                            found += 1
+
+                    if found == 1:
+                        j = subset_idx[0]
+                        if subset[j][indexB] != partBs[i]:
+                            subset[j][indexB] = partBs[i]
+                            subset[j][-1] += 1
+                            subset[j][-2] += candidate[partBs[i].astype(int), 2] + connection_all[k][i][2]
+                    elif found == 2:  # if found 2 and disjoint, merge them
+                        j1, j2 = subset_idx
+                        membership = ((subset[j1] >= 0).astype(int) + (subset[j2] >= 0).astype(int))[:-2]
+                        if len(np.nonzero(membership == 2)[0]) == 0:  # merge
+                            subset[j1][:-2] += (subset[j2][:-2] + 1)
+                            subset[j1][-2:] += subset[j2][-2:]
+                            subset[j1][-2] += connection_all[k][i][2]
+                            subset = np.delete(subset, j2, 0)
+                        else:  # as like found == 1
+                            subset[j1][indexB] = partBs[i]
+                            subset[j1][-1] += 1
+                            subset[j1][-2] += candidate[partBs[i].astype(int), 2] + connection_all[k][i][2]
+
+                    # if find no partA in the subset, create a new subset
+                    elif not found and k < 17:
+                        row = -1 * np.ones(20)
+                        row[indexA] = partAs[i]
+                        row[indexB] = partBs[i]
+                        row[-1] = 2
+                        row[-2] = sum(candidate[connection_all[k][i, :2].astype(int), 2]) + connection_all[k][i][2]
+                        subset = np.vstack([subset, row])
+        # delete some rows of subset which has few parts occur
+        deleteIdx = []
+        for i in range(len(subset)):
+            if subset[i][-1] < 4 or subset[i][-2] / subset[i][-1] < 0.4:
+                deleteIdx.append(i)
+        subset = np.delete(subset, deleteIdx, axis=0)
+
+        # subset: n*20 array, 0-17 is the index in candidate, 18 is the total score, 19 is the total parts
+        # candidate: x, y, score, id
+        return candidate, subset
+
+if __name__ == "__main__":
+    body_estimation = Body('../model/body_pose_model.pth')
+
+    test_image = '../images/ski.jpg'
+    oriImg = cv2.imread(test_image)  # B,G,R order
+    candidate, subset = body_estimation(oriImg)
+    canvas = util.draw_bodypose(oriImg, candidate, subset)
+    plt.imshow(canvas[:, :, [2, 1, 0]])
+    plt.show()

openpose-editor/scripts/openpose/model.py

@@ -0,0 +1,221 @@
+# This code from https://github.com/lllyasviel/ControlNet
+
+import torch
+from collections import OrderedDict
+
+import torch
+import torch.nn as nn
+
+def make_layers(block, no_relu_layers):
+    layers = []
+    for layer_name, v in block.items():
+        if 'pool' in layer_name:
+            layer = nn.MaxPool2d(kernel_size=v[0], stride=v[1],
+                                    padding=v[2])
+            layers.append((layer_name, layer))
+        else:
+            conv2d = nn.Conv2d(in_channels=v[0], out_channels=v[1],
+                               kernel_size=v[2], stride=v[3],
+                               padding=v[4])
+            layers.append((layer_name, conv2d))
+            if layer_name not in no_relu_layers:
+                layers.append(('relu_'+layer_name, nn.ReLU(inplace=True)))
+
+    return nn.Sequential(OrderedDict(layers))
+
+class bodypose_model(nn.Module):
+    def __init__(self):
+        super(bodypose_model, self).__init__()
+
+        # these layers have no relu layer
+        no_relu_layers = ['conv5_5_CPM_L1', 'conv5_5_CPM_L2', 'Mconv7_stage2_L1',\
+                          'Mconv7_stage2_L2', 'Mconv7_stage3_L1', 'Mconv7_stage3_L2',\
+                          'Mconv7_stage4_L1', 'Mconv7_stage4_L2', 'Mconv7_stage5_L1',\
+                          'Mconv7_stage5_L2', 'Mconv7_stage6_L1', 'Mconv7_stage6_L1']
+        blocks = {}
+        block0 = OrderedDict([
+                      ('conv1_1', [3, 64, 3, 1, 1]),
+                      ('conv1_2', [64, 64, 3, 1, 1]),
+                      ('pool1_stage1', [2, 2, 0]),
+                      ('conv2_1', [64, 128, 3, 1, 1]),
+                      ('conv2_2', [128, 128, 3, 1, 1]),
+                      ('pool2_stage1', [2, 2, 0]),
+                      ('conv3_1', [128, 256, 3, 1, 1]),
+                      ('conv3_2', [256, 256, 3, 1, 1]),
+                      ('conv3_3', [256, 256, 3, 1, 1]),
+                      ('conv3_4', [256, 256, 3, 1, 1]),
+                      ('pool3_stage1', [2, 2, 0]),
+                      ('conv4_1', [256, 512, 3, 1, 1]),
+                      ('conv4_2', [512, 512, 3, 1, 1]),
+                      ('conv4_3_CPM', [512, 256, 3, 1, 1]),
+                      ('conv4_4_CPM', [256, 128, 3, 1, 1])
+                  ])
+
+
+        # Stage 1
+        block1_1 = OrderedDict([
+                        ('conv5_1_CPM_L1', [128, 128, 3, 1, 1]),
+                        ('conv5_2_CPM_L1', [128, 128, 3, 1, 1]),
+                        ('conv5_3_CPM_L1', [128, 128, 3, 1, 1]),
+                        ('conv5_4_CPM_L1', [128, 512, 1, 1, 0]),
+                        ('conv5_5_CPM_L1', [512, 38, 1, 1, 0])
+                    ])
+
+        block1_2 = OrderedDict([
+                        ('conv5_1_CPM_L2', [128, 128, 3, 1, 1]),
+                        ('conv5_2_CPM_L2', [128, 128, 3, 1, 1]),
+                        ('conv5_3_CPM_L2', [128, 128, 3, 1, 1]),
+                        ('conv5_4_CPM_L2', [128, 512, 1, 1, 0]),
+                        ('conv5_5_CPM_L2', [512, 19, 1, 1, 0])
+                    ])
+        blocks['block1_1'] = block1_1
+        blocks['block1_2'] = block1_2
+
+        self.model0 = make_layers(block0, no_relu_layers)
+
+        # Stages 2 - 6
+        for i in range(2, 7):
+            blocks['block%d_1' % i] = OrderedDict([
+                    ('Mconv1_stage%d_L1' % i, [185, 128, 7, 1, 3]),
+                    ('Mconv2_stage%d_L1' % i, [128, 128, 7, 1, 3]),
+                    ('Mconv3_stage%d_L1' % i, [128, 128, 7, 1, 3]),
+                    ('Mconv4_stage%d_L1' % i, [128, 128, 7, 1, 3]),
+                    ('Mconv5_stage%d_L1' % i, [128, 128, 7, 1, 3]),
+                    ('Mconv6_stage%d_L1' % i, [128, 128, 1, 1, 0]),
+                    ('Mconv7_stage%d_L1' % i, [128, 38, 1, 1, 0])
+                ])
+
+            blocks['block%d_2' % i] = OrderedDict([
+                    ('Mconv1_stage%d_L2' % i, [185, 128, 7, 1, 3]),
+                    ('Mconv2_stage%d_L2' % i, [128, 128, 7, 1, 3]),
+                    ('Mconv3_stage%d_L2' % i, [128, 128, 7, 1, 3]),
+                    ('Mconv4_stage%d_L2' % i, [128, 128, 7, 1, 3]),
+                    ('Mconv5_stage%d_L2' % i, [128, 128, 7, 1, 3]),
+                    ('Mconv6_stage%d_L2' % i, [128, 128, 1, 1, 0]),
+                    ('Mconv7_stage%d_L2' % i, [128, 19, 1, 1, 0])
+                ])
+
+        for k in blocks.keys():
+            blocks[k] = make_layers(blocks[k], no_relu_layers)
+
+        self.model1_1 = blocks['block1_1']
+        self.model2_1 = blocks['block2_1']
+        self.model3_1 = blocks['block3_1']
+        self.model4_1 = blocks['block4_1']
+        self.model5_1 = blocks['block5_1']
+        self.model6_1 = blocks['block6_1']
+
+        self.model1_2 = blocks['block1_2']
+        self.model2_2 = blocks['block2_2']
+        self.model3_2 = blocks['block3_2']
+        self.model4_2 = blocks['block4_2']
+        self.model5_2 = blocks['block5_2']
+        self.model6_2 = blocks['block6_2']
+
+
+    def forward(self, x):
+
+        out1 = self.model0(x)
+
+        out1_1 = self.model1_1(out1)
+        out1_2 = self.model1_2(out1)
+        out2 = torch.cat([out1_1, out1_2, out1], 1)
+
+        out2_1 = self.model2_1(out2)
+        out2_2 = self.model2_2(out2)
+        out3 = torch.cat([out2_1, out2_2, out1], 1)
+
+        out3_1 = self.model3_1(out3)
+        out3_2 = self.model3_2(out3)
+        out4 = torch.cat([out3_1, out3_2, out1], 1)
+
+        out4_1 = self.model4_1(out4)
+        out4_2 = self.model4_2(out4)
+        out5 = torch.cat([out4_1, out4_2, out1], 1)
+
+        out5_1 = self.model5_1(out5)
+        out5_2 = self.model5_2(out5)
+        out6 = torch.cat([out5_1, out5_2, out1], 1)
+
+        out6_1 = self.model6_1(out6)
+        out6_2 = self.model6_2(out6)
+
+        return out6_1, out6_2
+
+class handpose_model(nn.Module):
+    def __init__(self):
+        super(handpose_model, self).__init__()
+
+        # these layers have no relu layer
+        no_relu_layers = ['conv6_2_CPM', 'Mconv7_stage2', 'Mconv7_stage3',\
+                          'Mconv7_stage4', 'Mconv7_stage5', 'Mconv7_stage6']
+        # stage 1
+        block1_0 = OrderedDict([
+                ('conv1_1', [3, 64, 3, 1, 1]),
+                ('conv1_2', [64, 64, 3, 1, 1]),
+                ('pool1_stage1', [2, 2, 0]),
+                ('conv2_1', [64, 128, 3, 1, 1]),
+                ('conv2_2', [128, 128, 3, 1, 1]),
+                ('pool2_stage1', [2, 2, 0]),
+                ('conv3_1', [128, 256, 3, 1, 1]),
+                ('conv3_2', [256, 256, 3, 1, 1]),
+                ('conv3_3', [256, 256, 3, 1, 1]),
+                ('conv3_4', [256, 256, 3, 1, 1]),
+                ('pool3_stage1', [2, 2, 0]),
+                ('conv4_1', [256, 512, 3, 1, 1]),
+                ('conv4_2', [512, 512, 3, 1, 1]),
+                ('conv4_3', [512, 512, 3, 1, 1]),
+                ('conv4_4', [512, 512, 3, 1, 1]),
+                ('conv5_1', [512, 512, 3, 1, 1]),
+                ('conv5_2', [512, 512, 3, 1, 1]),
+                ('conv5_3_CPM', [512, 128, 3, 1, 1])
+            ])
+
+        block1_1 = OrderedDict([
+            ('conv6_1_CPM', [128, 512, 1, 1, 0]),
+            ('conv6_2_CPM', [512, 22, 1, 1, 0])
+        ])
+
+        blocks = {}
+        blocks['block1_0'] = block1_0
+        blocks['block1_1'] = block1_1
+
+        # stage 2-6
+        for i in range(2, 7):
+            blocks['block%d' % i] = OrderedDict([
+                    ('Mconv1_stage%d' % i, [150, 128, 7, 1, 3]),
+                    ('Mconv2_stage%d' % i, [128, 128, 7, 1, 3]),
+                    ('Mconv3_stage%d' % i, [128, 128, 7, 1, 3]),
+                    ('Mconv4_stage%d' % i, [128, 128, 7, 1, 3]),
+                    ('Mconv5_stage%d' % i, [128, 128, 7, 1, 3]),
+                    ('Mconv6_stage%d' % i, [128, 128, 1, 1, 0]),
+                    ('Mconv7_stage%d' % i, [128, 22, 1, 1, 0])
+                ])
+
+        for k in blocks.keys():
+            blocks[k] = make_layers(blocks[k], no_relu_layers)
+
+        self.model1_0 = blocks['block1_0']
+        self.model1_1 = blocks['block1_1']
+        self.model2 = blocks['block2']
+        self.model3 = blocks['block3']
+        self.model4 = blocks['block4']
+        self.model5 = blocks['block5']
+        self.model6 = blocks['block6']
+
+    def forward(self, x):
+        out1_0 = self.model1_0(x)
+        out1_1 = self.model1_1(out1_0)
+        concat_stage2 = torch.cat([out1_1, out1_0], 1)
+        out_stage2 = self.model2(concat_stage2)
+        concat_stage3 = torch.cat([out_stage2, out1_0], 1)
+        out_stage3 = self.model3(concat_stage3)
+        concat_stage4 = torch.cat([out_stage3, out1_0], 1)
+        out_stage4 = self.model4(concat_stage4)
+        concat_stage5 = torch.cat([out_stage4, out1_0], 1)
+        out_stage5 = self.model5(concat_stage5)
+        concat_stage6 = torch.cat([out_stage5, out1_0], 1)
+        out_stage6 = self.model6(concat_stage6)
+        return out_stage6
+
+