File size: 990 Bytes
0e83a2b
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Mettle usage
============

Install the dependencies in requirements.txt, then load a local checkout or
Hugging Face repository with the standard Transformers auto classes:

    from PIL import Image
    from transformers import AutoImageProcessor, AutoModel

    repo = "slideflow-labs/Mettle"
    processor = AutoImageProcessor.from_pretrained(repo)
    model = AutoModel.from_pretrained(
        repo,
        trust_remote_code=True,
    ).eval()

    image = Image.open("tile.png").convert("RGB")
    pixel_values = processor(images=image, return_tensors="pt").pixel_values

    # Public benchmark representation: (batch, 3072)
    embedding = model.encode(pixel_values, feature_view="cls_mean")

    # Compatibility representation: (batch, 1536)
    cls_embedding = model.encode(pixel_values, feature_view="cls")

The first 1536 coordinates of cls_mean are exactly cls_embedding. The remaining
1536 coordinates are the mean of spatial patch tokens; CLS and register tokens
are excluded.