IMvision12 commited on
Commit
2cfe427
·
verified ·
1 Parent(s): df85c7a

fix readme.md

Browse files
Files changed (1) hide show
  1. README.md +66 -15
README.md CHANGED
@@ -1,30 +1,81 @@
1
  ---
2
  pipeline_tag: image-classification
3
  license: other
4
- license_name: nvidia-segformer-license
5
- license_link: https://github.com/NVlabs/SegFormer/blob/master/LICENSE
6
  base_model: nvidia/mit-b3
7
  library_name: kerasformers
8
  tags:
9
- - keras
10
- - kerasformers
11
- - image-classification
12
- - mit
13
- - pytorch
14
- - jax
15
- - tf
 
 
16
  ---
17
 
18
- # mit_b3_in1k
19
 
20
- Pure-Keras 3 image-classification weight for [kerasformers](https://github.com/IMvision12/KerasFormers), converted from [nvidia/mit-b3](https://huggingface.co/nvidia/mit-b3).
21
 
 
22
 
23
- ## Usage
 
 
 
 
 
 
 
 
 
 
 
 
24
 
25
  ```python
26
- from kerasformers.models.mit import MiTImageClassify
27
- model = MiTImageClassify.from_weights("mit_b3_in1k")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  ```
29
 
30
- License: **other** (nvidia-segformer-license), inherited from the upstream source.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  pipeline_tag: image-classification
3
  license: other
 
 
4
  base_model: nvidia/mit-b3
5
  library_name: kerasformers
6
  tags:
7
+ - keras
8
+ - kerasformers
9
+ - image-classification
10
+ - mit
11
+ - backbone
12
+ - arxiv:2105.15203
13
+ - pytorch
14
+ - jax
15
+ - tf
16
  ---
17
 
18
+ ## ***See [our collection](https://huggingface.co/collections/kerasformers/mit-segformer-encoder-6a6e81367fda42bf79b426e8) for all versions of MiT.***
19
 
20
+ # Run MiT with Keras 3: JAX, PyTorch, or TensorFlow
21
 
22
+ [![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-black?logo=github)](https://github.com/IMvision12/KerasFormers) [![Docs](https://img.shields.io/badge/Docs-MiT-blue)](https://imvision12.github.io/KerasFormers/classification_backbones/) [![Collection](https://img.shields.io/badge/HF-MiT%20collection-yellow)](https://huggingface.co/collections/kerasformers/mit-segformer-encoder-6a6e81367fda42bf79b426e8)
23
 
24
+ # kerasformers/mit_b3_in1k
25
+
26
+ Paper: [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers (arXiv:2105.15203)](https://arxiv.org/abs/2105.15203) · [HF Papers](https://huggingface.co/papers/2105.15203)
27
+
28
+ MiT is the hierarchical Mix Transformer encoder from SegFormer, also usable for ImageNet classification. For full SegFormer segmentation heads, see the SegFormer collection.
29
+
30
+ For more details on the model, please go to the upstream [model card](https://huggingface.co/nvidia/mit-b3).
31
+
32
+ Pure-**Keras 3** conversion of [`nvidia/mit-b3`](https://huggingface.co/nvidia/mit-b3) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
33
+
34
+ This is an **image-classification / backbone** checkpoint (`MiTImageClassify` / `MiTModel`).
35
+
36
+ ## ✨ Quick start
37
 
38
  ```python
39
+ import os
40
+ os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
41
+
42
+ from PIL import Image
43
+ import numpy as np
44
+ from kerasformers.models.mit import MiTImageClassify, MiTModel
45
+
46
+ model = MiTImageClassify.from_weights("kerasformers/mit_b3_in1k")
47
+ backbone = MiTModel.from_weights(
48
+ "kerasformers/mit_b3_in1k", as_backbone=True
49
+ )
50
+
51
+ image = Image.open("your_image.jpg").convert("RGB")
52
+ image = image.resize((224, 224))
53
+ x = np.asarray(image, dtype="float32")[None] # (1, H, W, 3)
54
+ print(model(x).shape) # (1, num_classes)
55
+ feats = backbone(x)
56
+ print(len(feats), [tuple(f.shape) for f in feats])
57
  ```
58
 
59
+ Load any MiT variant the same way with `from_weights("kerasformers/<variant>")`:
60
+
61
+ | Variant | Hub |
62
+ |---|---|
63
+ | `mit_b0_in1k` | [`kerasformers/mit_b0_in1k`](https://huggingface.co/kerasformers/mit_b0_in1k) |
64
+ | `mit_b1_in1k` | [`kerasformers/mit_b1_in1k`](https://huggingface.co/kerasformers/mit_b1_in1k) |
65
+ | `mit_b2_in1k` | [`kerasformers/mit_b2_in1k`](https://huggingface.co/kerasformers/mit_b2_in1k) |
66
+ | `mit_b3_in1k` | [`kerasformers/mit_b3_in1k`](https://huggingface.co/kerasformers/mit_b3_in1k) |
67
+ | `mit_b4_in1k` | [`kerasformers/mit_b4_in1k`](https://huggingface.co/kerasformers/mit_b4_in1k) |
68
+ | `mit_b5_in1k` | [`kerasformers/mit_b5_in1k`](https://huggingface.co/kerasformers/mit_b5_in1k) |
69
+
70
+ ## Tips
71
+
72
+ - Set `KERAS_BACKEND` **before** importing Keras / kerasformers.
73
+ - `MiTImageClassify` returns class logits; `MiTModel` returns features (`as_backbone=True` for multi-scale stages).
74
+ - See [docs](https://imvision12.github.io/KerasFormers/classification_backbones/) and [Loading Weights](https://imvision12.github.io/KerasFormers/loading_weights/).
75
+ - Upstream / timm checkpoints: `MiTImageClassify.from_weights("hf:nvidia/mit-b3")`.
76
+
77
+ ## Special Thanks
78
+
79
+ A huge thank you to the MiT authors and the timm / Hub communities for creating and releasing these models.
80
+
81
+ License: see YAML `license` (usually matches the upstream checkpoint).