VictorButoi commited on
Commit
2cfa0f8
·
verified ·
1 Parent(s): 92326cf

Remove old template readme

Browse files
Files changed (1) hide show
  1. README.md +0 -99
README.md CHANGED
@@ -1,99 +0,0 @@
1
- ---
2
- license: apache-2.0
3
- pipeline_tag: mask-generation
4
- library_name: transformers
5
- ---
6
-
7
- CURRENT COPIED FROM SAM REPO IN PROGRESS
8
-
9
- Repository for FleXray: Flexible Full-Body X-ray Segmentation. See the [FleXray paper](https://arxiv.org/abs/0613.2000) for more information.
10
-
11
- The official training code is publicly release in this [repo](https://github.com/VictorButoi/FleXray/).
12
-
13
- ## Usage
14
-
15
- For image prediction:
16
-
17
- ```python
18
- import torch
19
- from sam2.sam2_image_predictor import SAM2ImagePredictor
20
- predictor = SAM2ImagePredictor.from_pretrained("facebook/sam2-hiera-large")
21
- with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
22
- predictor.set_image(<your_image>)
23
- masks, _, _ = predictor.predict(<input_prompts>)
24
- ```
25
-
26
- For video prediction:
27
-
28
- ```python
29
- import torch
30
- from sam2.sam2_video_predictor import SAM2VideoPredictor
31
- predictor = SAM2VideoPredictor.from_pretrained("facebook/sam2-hiera-large")
32
- with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
33
- state = predictor.init_state(<your_video>)
34
- # add new prompts and instantly get the output on the same frame
35
- frame_idx, object_ids, masks = predictor.add_new_points_or_box(state, <your_prompts>):
36
- # propagate the prompts to get masklets throughout the video
37
- for frame_idx, object_ids, masks in predictor.propagate_in_video(state):
38
- ...
39
- ```
40
-
41
- Refer to the [demo notebooks](https://github.com/facebookresearch/segment-anything-2/tree/main/notebooks) for details.
42
-
43
- ## Usage with 🤗 Transformers
44
-
45
-
46
- ### Automatic Mask Generation with Pipeline
47
-
48
- SAM2 can be used for automatic mask generation to segment all objects in an image using the `mask-generation` pipeline:
49
-
50
- ```python
51
- >>> from transformers import pipeline
52
- >>> generator = pipeline("mask-generation", model="facebook/sam2-hiera-large", device=0)
53
- >>> image_url = "https://huggingface.co/datasets/hf-internal-testing/sam2-fixtures/resolve/main/truck.jpg"
54
- >>> outputs = generator(image_url, points_per_batch=64)
55
- >>> len(outputs["masks"]) # Number of masks generated
56
- 39
57
- ```
58
-
59
- ### Basic Image Segmentation
60
-
61
- #### Single Point Click
62
-
63
- You can segment objects by providing a single point click on the object you want to segment:
64
-
65
- ```python
66
- >>> from transformers import Sam2Processor, Sam2Model
67
- >>> import torch
68
- >>> from PIL import Image
69
- >>> import requests
70
- >>> device = "cuda" if torch.cuda.is_available() else "cpu"
71
- >>> model = Sam2Model.from_pretrained("facebook/sam2-hiera-large").to(device)
72
- >>> processor = Sam2Processor.from_pretrained("facebook/sam2-hiera-large")
73
- >>> image_url = "https://huggingface.co/datasets/hf-internal-testing/sam2-fixtures/resolve/main/truck.jpg"
74
- >>> raw_image = Image.open(requests.get(image_url, stream=True).raw).convert("RGB")
75
- >>> input_points = [[[[500, 375]]]] # Single point click, 4 dimensions (image_dim, object_dim, point_per_object_dim, coordinates)
76
- >>> input_labels = [[[1]]] # 1 for positive click, 0 for negative click, 3 dimensions (image_dim, object_dim, point_label)
77
- >>> inputs = processor(images=raw_image, input_points=input_points, input_labels=input_labels, return_tensors="pt").to(device)
78
- >>> with torch.no_grad():
79
- ... outputs = model(**inputs)
80
- >>> masks = processor.post_process_masks(outputs.pred_masks.cpu(), inputs["original_sizes"])[0]
81
- >>> # The model outputs multiple mask predictions ranked by quality score
82
- >>> print(f"Generated {masks.shape[1]} masks with shape {masks.shape}")
83
- Generated 3 masks with shape torch.Size(1, 3, 1500, 2250)
84
- ```
85
-
86
-
87
- ### Citation
88
-
89
- To cite the paper, model, or software, please use the below:
90
- ```
91
- @article{ravi2024sam2,
92
- title={SAM 2: Segment Anything in Images and Videos},
93
- author={Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and R{\"a}dle, Roman and Rolland, Chloe and Gustafson, Laura and Mintun, Eric and Pan, Junting and Alwala, Kalyan Vasudev and Carion, Nicolas and Wu, Chao-Yuan and Girshick, Ross and Doll{\'a}r, Piotr and Feichtenhofer, Christoph},
94
- journal={arXiv preprint arXiv:2408.00714},
95
- url={https://arxiv.org/abs/2408.00714},
96
- year={2024}
97
- }
98
- ```
99
-