Update locon/notes.md

locon/notes.md

@@ -0,0 +1,21 @@
+# LoCon Testing
+https://github.com/KohakuBlueleaf/LoCon
+
+## Dim/Alpha
+Testing with
+conv: 1dim 1alpha (due to coding error the examples are using 4alpha, has since been fixed in the code base)
+linear(network): 4dim 4alpha
+We will refer to dim and alpha in the format of dim/alpha, ie 1/1, 4/4
+
+## Training Time
+Character appears to gain significant accuracy sooner than using Lora (600 steps vs 800). The time per step is a little bit slower but a better analysis of this is required.
+We suspect that we can train characters faster in terms of real time.
+
+## Mixing
+Applying style on character appears to work well. The example wlop style applied on Amber is 1/0.25 and 4/1. The character loses some stability but the style LoCon can easily be lowered and still retain great style retention while significantly reducing the impact to character correctness.
+Overall it appears that mixing is more stable with LoCon than lora but this could be due to the dim/alpha and training settings between these two locon being closer compared to LoRA land where popular dim sizes range from 8 to 128.
+
+## Finetune model extractions
+To be tested. We suspect the performance should be much better than LoRA extraction since all the model will be extracted from.
+https://github.com/KohakuBlueleaf/LoCon/blob/main/extract_locon.py
+