Commit
·
2b7de13
1
Parent(s):
a47a1fd
Update README.md
Browse files
README.md
CHANGED
|
@@ -8,13 +8,9 @@ tags:
|
|
| 8 |
|
| 9 |
|
| 10 |
## Introduction
|
| 11 |
-
|
| 12 |
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
Integrating the strengths of both ControlNet and PEFT, this approach offers a more efficient and compact method to bring model control for a wider variety of consumer GPUs.
|
| 16 |
-
|
| 17 |
-
For each model below, you'll find `Rank 256` files (reducing the `~4.7GB` ControlNets to `~738MB`) and experimental, ultra-pruned `Rank 128` files (reducing to `~377MB`).
|
| 18 |
|
| 19 |
Each Control-LoRA has been trained on a diverse range of image concepts and aspect ratios.
|
| 20 |
### MiDaS and ClipDrop Depth
|
|
@@ -22,15 +18,15 @@ Each Control-LoRA has been trained on a diverse range of image concepts and aspe
|
|
| 22 |
|
| 23 |
Depth estimation is an image processing technique that determines the distance of objects in a scene, providing a depth map that highlights variations in proximity.
|
| 24 |
|
| 25 |
-
In the example above, we compare the depth results of MiDaS dpt_beit_large_512 with ClipDrop Depth for portraits, and their subsequent use in Depth Control-LoRa.
|
| 26 |
-
|
| 27 |
The Control-LoRA utilizes a grayscale depth map for guided generation.
|
| 28 |
|
|
|
|
|
|
|
| 29 |
### Canny Edge
|
| 30 |

|
| 31 |
Canny Edge Detection is an image processing technique that identifies abrupt changes in intensity to highlight edges in an image.
|
| 32 |
|
| 33 |
-
This Control-LoRA uses the edges from an image to
|
| 34 |
|
| 35 |
### Photograph and Sketch Colorizer
|
| 36 |

|
|
|
|
| 8 |
|
| 9 |
|
| 10 |
## Introduction
|
| 11 |
+
By adding low-rank parameter efficient fine tuning to ControlNet, we introduce ***Control-LoRAs***. This approach offers a more efficient and compact method to bring model control to a wider variety of consumer GPUs.
|
| 12 |
|
| 13 |
+
For each model below, you'll find `Rank 256` files (reducing the `~4.7GB` ControlNets to `~738MB`) and experimental, `Rank 128` files (reduced to `~377MB`).
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
Each Control-LoRA has been trained on a diverse range of image concepts and aspect ratios.
|
| 16 |
### MiDaS and ClipDrop Depth
|
|
|
|
| 18 |
|
| 19 |
Depth estimation is an image processing technique that determines the distance of objects in a scene, providing a depth map that highlights variations in proximity.
|
| 20 |
|
|
|
|
|
|
|
| 21 |
The Control-LoRA utilizes a grayscale depth map for guided generation.
|
| 22 |
|
| 23 |
+
In the example above, we compare the depth results of `MiDaS dpt_beit_large_512` and the `Portrait Depth Estimation` (available in the [ClipDrop API by Stability AI](https://clipdrop.co/apis/docs/portrait-depth-estimation)).
|
| 24 |
+
|
| 25 |
### Canny Edge
|
| 26 |

|
| 27 |
Canny Edge Detection is an image processing technique that identifies abrupt changes in intensity to highlight edges in an image.
|
| 28 |
|
| 29 |
+
This Control-LoRA uses the edges from an image to generate the final image.
|
| 30 |
|
| 31 |
### Photograph and Sketch Colorizer
|
| 32 |

|