Hugging Face's logo

Spaces:
touhou-ai-experimental
/
research-paper
Running

App Files Community
parsee-mizuhashi commited on
Commit
80949ae
·
1 Parent(s): 6975069

Upload 2 files

Browse files
Files changed (2) hide show
  1. index.html +92 -17
  2. style.css +123 -18
index.html CHANGED
@@ -1,19 +1,94 @@
1
  <!DOCTYPE html>
2
- <html>
3
- <head>
4
- <meta charset="utf-8" />
5
- <meta name="viewport" content="width=device-width" />
6
- <title>My static Space</title>
7
- <link rel="stylesheet" href="style.css" />
8
- </head>
9
- <body>
10
- <div class="card">
11
- <h1>Welcome to your static Space!</h1>
12
- <p>You can modify this app directly by editing <i>index.html</i> in the Files and versions tab.</p>
13
- <p>
14
- Also don't forget to check the
15
- <a href="https://huggingface.co/docs/hub/spaces" target="_blank">Spaces documentation</a>.
16
- </p>
17
- </div>
18
- </body>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  </html>
 
1
  <!DOCTYPE html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="UTF-8">
5
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
6
+ <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.3/css/all.min.css">
7
+ <link href="https://fonts.googleapis.com/css2?family=Montserrat:wght@500&display=swap" rel="stylesheet">
8
+ <link rel="stylesheet" href="style.css">
9
+ <title>M.o.f.u.</title>
10
+ </head>
11
+ <body>
12
+ <h1 class="header-title">M.o.f.u.</h1>
13
+ <p class="header-subtitle"><span class="highlight-orange">Mo</span>del independent, <span class="highlight-violate">F</span>ast T<span class="highlight-orange">u</span>ning of Stable Diffusion concepts</p>
14
+ <section id="abstract">
15
+ <h2><i class="icon fas fa-file-alt"></i> Abstract</h2>
16
+ <p>I present MoFu, a model-independent, fast tuning
17
+ approach that enhances Stable Diffusion. Compared
18
+ to other more traditional methods, such as Low Rank
19
+ adaptation for the model or fine tuning it, MoFu
20
+ doesn’t modify the weights of the main model at all.
21
+ MoFu seamlessly integrates with Stable Diffusion's
22
+ text encoder, enabling rapid style/concept addition
23
+ without modifying or fine-tuning the encoder's
24
+ weights</p>
25
+ </section>
26
+
27
+ <section id="methodology">
28
+ <h2><i class="icon fas fa-flask"></i> Methodology</h2>
29
+ <p>The methodology of MoFu revolves around a simple
30
+ yet effective process. To achieve the desired results,
31
+ we begin by comparing natural prompts given to a
32
+ set of images. This comparison allows us to extract
33
+ the essential concepts or styles from the text
34
+ prompts. These identified concepts are then stored in
35
+ a mixin, creating a compact representation of the
36
+ desired style information. The mixin is designed to be
37
+ compatible with Stable Diffusion's architecture and
38
+ serves as an additive to the text encoder’s output.
39
+ By adding the mixin with the text encoder’s output
40
+ (the mixin, or MoFu model, can also be multiplied by
41
+ a weight, in order to make its effect stronger or
42
+ weaker), MoFu efficiently injects the extracted
43
+ concepts into the image generation process. This
44
+ injection enables Stable Diffusion to generate images
45
+ with the desired style without altering the underlying
46
+ weights of the main model. As a result, MoFu
47
+ provides a powerful and flexible solution for style
48
+ transfer or concept addition in Stable Diffusion
49
+ without the need for extensive model modifications
50
+ or resource-intensive fine-tuning.</p>
51
+ </section>
52
+
53
+ <section id="results">
54
+ <h2><i class="icon fas fa-chart-bar"></i> Results</h2>
55
+ <p>To evaluate the effectiveness of MoFu, I conducted a
56
+ series of experiments and compared its performance
57
+ to LoRA and fine-tuning methods. Our results
58
+ demonstrate that MoFu achieves comparable
59
+ performance to LoRAs while requiring significantly
60
+ less training time, taking only around 10-20 seconds
61
+ on average, primarily due to being CPU-bound. This
62
+ is in stark contrast to LoRAs, which typically demand
63
+ several hours to train. However, I also observed that
64
+ MoFu falls short of fine-tuning, as the latter can
65
+ achieve even better precision/quality but at the cost
66
+ of a much longer training.
67
+ </p>
68
+ </section>
69
+
70
+ <section id="conclusion">
71
+ <h2><i class="icon fas fa-clipboard-check"></i> Conclusion</h2>
72
+ <p>In conclusion, MoFu offers an efficient and
73
+ model-independent solution for adding new styles or
74
+ concepts to Stable Diffusion without modifying the
75
+ main model's weights. It achieves comparable results
76
+ to LoRA while significantly reducing training time,
77
+ making it a practical choice for rapid adaptation.
78
+ Though fine-tuning still outperforms MoFu in quality,
79
+ the trade-off between speed and accuracy makes
80
+ MoFu a valuable option for various applications.
81
+ Future work may focus on optimizing the
82
+ implementation / quality of MoFu.</p>
83
+ </section>
84
+
85
+ <footer>
86
+ <div class="buttons">
87
+ <a href="mailto:parsee.mizuhashi.th11@gmail.com" class="button">Yoinked</a>
88
+ <a href="https://huggingface.co/organizations/touhou-ai-experimental" class="button">Touhou AI Experimental Group</a>
89
+ <a href="https://huggingface.co/mofu-team" class="button">MoFu</a>
90
+ <a href="https://github.com/yoinked-h/MoFu" class="button" target="_blank">GitHub</a>
91
+ </div>
92
+ </footer>
93
+ </body>
94
  </html>
style.css CHANGED
@@ -1,28 +1,133 @@
 
 
 
 
 
 
1
  body {
2
- padding: 2rem;
3
- font-family: -apple-system, BlinkMacSystemFont, "Arial", sans-serif;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  }
5
 
6
- h1 {
7
- font-size: 16px;
8
- margin-top: 0;
 
 
 
9
  }
10
 
11
- p {
12
- color: rgb(107, 114, 128);
13
- font-size: 15px;
14
- margin-bottom: 10px;
15
- margin-top: 5px;
16
  }
17
 
18
- .card {
19
- max-width: 620px;
20
- margin: 0 auto;
21
- padding: 16px;
22
- border: 1px solid lightgray;
23
- border-radius: 16px;
24
  }
25
 
26
- .card p:last-child {
27
- margin-bottom: 0;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  }
 
1
+ /* Reset some default styling */
2
+ body, h1, h2, p {
3
+ margin: 0;
4
+ padding: 0;
5
+ }
6
+
7
  body {
8
+ font-family: Arial, sans-serif;
9
+ background-color: #121212;
10
+ color: #FFFFFF;
11
+ }
12
+
13
+ .header-title {
14
+ font-family: 'Montserrat', sans-serif;
15
+ font-size: 24px;
16
+ text-align: center;
17
+ color: #FF4500; /* Original Orange-Red Color */
18
+ margin: 0;
19
+ padding: 20px 0;
20
+ }
21
+ .header-subtitle {
22
+ font-size: 16px;
23
+ text-align: center;
24
+ color: #FF4500;
25
+ margin: 10px 0;
26
+ }
27
+
28
+ .highlight-orange {
29
+ color: #EE82EE; /* Original Orange-Red Color */
30
+ }
31
+
32
+ .highlight-violate {
33
+ color: #EE82EE; /* Violate Color */
34
+ }
35
+ header {
36
+ display: flex;
37
+ justify-content: space-between;
38
+ align-items: center;
39
+ background-color: #FF4500; /* Original Orange-Red Color */
40
+ padding: 10px 20px;
41
+ }
42
+
43
+ nav ul {
44
+ list-style-type: none;
45
+ display: flex;
46
+ }
47
+
48
+ nav li {
49
+ margin-right: 20px;
50
+ }
51
+
52
+ nav a {
53
+ text-decoration: none;
54
+ color: #FFFFFF;
55
  }
56
 
57
+ section {
58
+ background-color: #1E1E1E;
59
+ border: 2px solid #FF4500;
60
+ border-radius: 10px;
61
+ padding: 20px;
62
+ margin: 20px;
63
  }
64
 
65
+ section:hover {
66
+ border-color: #EE82EE;
 
 
 
67
  }
68
 
69
+ h2 {
70
+ color: #FF4500;
71
+ margin-bottom: 10px;
 
 
 
72
  }
73
 
74
+ .icon {
75
+ margin-right: 10px;
76
+ }
77
+
78
+ .footer {
79
+ text-align: center;
80
+ margin-top: 40px;
81
+ }
82
+
83
+ .buttons {
84
+ display: flex;
85
+ justify-content: center;
86
+ align-items: center;
87
+ margin: 20px auto;
88
+ }
89
+
90
+ .button {
91
+ background-color: #FF4500;
92
+ color: #121212;
93
+ padding: 8px 16px;
94
+ border-radius: 10px;
95
+ text-decoration: none;
96
+ font-weight: bold;
97
+ transition: background-color 0.3s;
98
+ margin-right: 10px;
99
+ }
100
+
101
+ .button:last-child {
102
+ margin-right: 0;
103
+ }
104
+
105
+ .button:hover {
106
+ background-color: #EE82EE;
107
+ }
108
+
109
+ /* Media Queries for Responsive Design */
110
+ @media (max-width: 768px) {
111
+ header {
112
+ padding: 10px;
113
+ }
114
+
115
+ nav ul {
116
+ flex-direction: row;
117
+ }
118
+
119
+ nav li {
120
+ margin-right: 10px;
121
+ }
122
+
123
+ .buttons {
124
+ flex-direction: column;
125
+ align-items: center;
126
+ margin: 10px auto;
127
+ }
128
+
129
+ .button {
130
+ margin-right: 0;
131
+ margin-bottom: 10px;
132
+ }
133
  }