samsartor commited on
Commit
1e50807
·
verified ·
1 Parent(s): 4143dfe

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +128 -3
README.md CHANGED
@@ -1,3 +1,128 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model:
4
+ - stabilityai/stable-diffusion-xl-base-1.0
5
+ - stabilityai/stable-diffusion-3-medium
6
+ - black-forest-labs/FLUX.1-dev
7
+ pipeline_tag: image-to-image
8
+ tags:
9
+ - Teamwork
10
+ - Intrinsic
11
+ - Inpainting
12
+ - SVBRDF
13
+ ---
14
+
15
+ <h1 align="center">Teamwork - Models Release</h1>
16
+ <p align="center">
17
+ <a href="https://bin.samsartor.com/teamwork.pdf">📃 Paper</a> | <a href="https://samsartor.com/teamwork">🌐 Website</a> | <a href="https://bin.samsartor.com/teamwork_supplemental.pdf">🖼️ Supplemental</a>
18
+ </p>
19
+ <p align="center"><img width="700px" src="https://samsartor.com/teamwork_teaser.svg" /></p>
20
+
21
+ Models presented in the paper "Teamwork: Collaborative Diffusion with Low-rank Coordination and Adaptation".
22
+
23
+ Use with https://github.com/samsartor/teamwork.
24
+
25
+ ```python
26
+ from teamwork.pipelines import TeamworkPipeline
27
+ from PIL import Image
28
+
29
+ pipe = TeamworkPipeline.from_checkpoint(
30
+ 'samsartor/teamwork-release',
31
+ 'decomposition_heterogeneous_sd3.safetensors',
32
+ ).to('cuda')
33
+
34
+ generated = pipe({'image': Image.open('./demo/red_glass_sphere.png'})
35
+ ```
36
+
37
+ Teamwork's safetensors files contain (roughly) the parameters of LoRAs to apply to a base model. However, in addition
38
+ to adapting the base model for a new task, Teamwork enables the model to accept any number of input images
39
+ and then collaboratively generate any number of outputs. The available models are:
40
+
41
+ <table>
42
+ <th><td>Description</td><td>Base Model</td><td>Resolution</td><td>Inputs</td><td>Outputs</td></th>
43
+ <tr>
44
+ <td>decomposition_heterogeneous_sd3.safetensors</td>
45
+ <td>Intrinsic image decomposition trained on heterogeneous datasets<br>(InteriorVerse, HyperSim, CGIntrinsics, Infinigen, ABC).</td>
46
+ <td>SD3</td>
47
+ <td>1024x1024</td>
48
+ <td>image</td>
49
+ <td>diffuse<br>specular<br>roughness<br>normals<br>depth<br>albedo<br>inverseshading<br>diffuseshading<br>residual</td>
50
+ </tr>
51
+ <tr>
52
+ <td>decomposition_interiorverse_sd3.safetensors</td>
53
+ <td>Intrinsic image decomposition specialized for InteriorVerse dataset.</td>
54
+ <td>SD3</td>
55
+ <td>640x480</td>
56
+ <td>image</td>
57
+ <td>diffuse<br>specular<br>roughness<br>normals<br>depth<br>albedo<br>inverseshading</td>
58
+ </tr>
59
+ <tr>
60
+ <td>inpainting_sd3.safetensors</td>
61
+ <td>Text-conditional inpainting of masked regions.</td>
62
+ <td>SD3</td>
63
+ <td>1024x1024</td>
64
+ <td>mask<br>masked</td>
65
+ <td>image</td>
66
+ </tr>
67
+ <tr>
68
+ <td>neuralshading_sd3.safetensors</td>
69
+ <td>Synthesizes images from intrinsic components under unknown random lighting.</td>
70
+ <td>SD3</td>
71
+ <td>1024x1024</td>
72
+ <td>albedo<br>diffuse<br>specular<br>roughness<br>normals<br>depth</td>
73
+ <td>image<br>diffuseshading<br>inverseshading<br>residual</td>
74
+ </tr>
75
+ <tr>
76
+ <td>colocated_svbrdf_estimation_flux_256.safetensors</td>
77
+ <td>Estimates material properties from a single flash-lit photograph.</td>
78
+ <td>FLUX.1-dev</td>
79
+ <td>256x256</td>
80
+ <td>image<br>halfway</td>
81
+ <td>diffuse<br>specular<br>roughness<br>normals</td>
82
+ </tr>
83
+ <tr>
84
+ <td>colocated_svbrdf_estimation_flux_512.safetensors</td>
85
+ <td>Estimates material properties from a single flash-lit photograph.</td>
86
+ <td>FLUX.1-dev</td>
87
+ <td>512x512</td>
88
+ <td>image<br>halfway</td>
89
+ <td>diffuse<br>specular<br>roughness<br>normals</td>
90
+ </tr>
91
+ <tr>
92
+ <td>colocated_svbrdf_estimation_sd3_256.safetensors</td>
93
+ <td>Estimates material properties from a single flash-lit photograph.</td>
94
+ <td>SD3</td>
95
+ <td>256x256</td>
96
+ <td>image<br>halfway</td>
97
+ <td>diffuse<br>specular<br>roughness<br>normals</td>
98
+ </tr>
99
+ <tr>
100
+ <td>colocated_svbrdf_estimation_sd3_512.safetensors</td>
101
+ <td>Estimates material properties from a single flash-lit photograph.</td>
102
+ <td>SD3</td>
103
+ <td>512x512</td>
104
+ <td>image<br>halfway</td>
105
+ <td>diffuse<br>specular<br>roughness<br>normals</td>
106
+ </tr>
107
+ <tr>
108
+ <td>colocated_svbrdf_estimation_sdxl_256.safetensors</td>
109
+ <td>Estimates material properties from a single flash-lit photograph.</td>
110
+ <td>SDXL</td>
111
+ <td>256x256</td>
112
+ <td>image<br>halfway</td>
113
+ <td>diffuse<br>specular<br>roughness<br>normals</td>
114
+ </tr>
115
+ </table>
116
+
117
+
118
+ ## Citation
119
+
120
+ ```
121
+ @conference{Sartor:2025:TCD,
122
+ author = {Sartor, Sam and Peers, Pieter},
123
+ title = {Teamwork: Collaborative Diffusion with Low-rank Coordination and Adaptation},
124
+ month = {December},
125
+ year = {2025},
126
+ booktitle = {ACM SIGGRAPH Asia Conference Proceedings},
127
+ }
128
+ ```