RiddleHe commited on
Commit
a6ebcc2
·
verified ·
1 Parent(s): 79a1e19

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -2
README.md CHANGED
@@ -33,7 +33,21 @@ prompt: An image of breast cancer histopathology with detailed cellular structur
33
  #### How to use
34
 
35
  ```python
36
- # TODO: add an example code snippet for running this diffusion pipeline
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  ```
38
 
39
  #### Limitations and bias
@@ -42,4 +56,6 @@ prompt: An image of breast cancer histopathology with detailed cellular structur
42
 
43
  ## Training details
44
 
45
- [TODO: describe the data used to train the model]
 
 
 
33
  #### How to use
34
 
35
  ```python
36
+ controlnet = ControlNetModel.from_pretrained("RiddleHe/SD14_pathology_controlnet", torch_dtype=torch.float16)
37
+ pipe = StableDiffusionControlNetPipeline.from_pretrained(
38
+ "CompVis/stable-diffusion-v1-4", controlnet=controlnet, torch_dtype=torch.float16
39
+ )
40
+ pipe.load_lora_weights("RiddleHe/SD14_pathology_lora") # Load lora weights for pathology image generation
41
+ pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config)
42
+ pipe.to('cuda')
43
+
44
+ prompt = "A histopathology image of breast cancer tissue."
45
+ mask = mask.convert("RGB") # Provide a mask
46
+
47
+ generator = torch.Generator(device='cuda').manual_seed(42)
48
+
49
+ with torch.no_grad():
50
+ out = pipe(prompt, image=mask, num_inference_steps=70, num_images_per_prompt=3, generator=generator).images
51
  ```
52
 
53
  #### Limitations and bias
 
56
 
57
  ## Training details
58
 
59
+ The model is trained on 28216 image-mask pairs from the BRCA breast cancer dataset. Input is mask and output is image.
60
+
61
+ Mask is a single channel image with integer values from 0 to 21 representing 22 classes, eg. 1 representing tumor, 2 representing stroma.