Hugging Face's logo

zeromodels
/
qwen2.5-vl-72b-instruct

Image-Text-to-Text
Keras
PyTorch
JAX
TensorFlow
English
zeromodels
qwen2_5_vl
qwen2.5-vl
multimodal
vision
Model card Files
xet
 Community
IMvision12 commited on
Commit
6cf2bc4
·
0 Parent(s):

Super-squash branch 'main' using huggingface_hub

Browse files
Files changed (34) hide show
  1. .gitattributes +35 -0
  2. README.md +108 -0
  3. kf_config.json +50 -0
  4. kf_preprocessor.json +22 -0
  5. model.weights.json +0 -0
  6. model_00000.weights.h5 +3 -0
  7. model_00001.weights.h5 +3 -0
  8. model_00002.weights.h5 +3 -0
  9. model_00003.weights.h5 +3 -0
  10. model_00004.weights.h5 +3 -0
  11. model_00005.weights.h5 +3 -0
  12. model_00006.weights.h5 +3 -0
  13. model_00007.weights.h5 +3 -0
  14. model_00008.weights.h5 +3 -0
  15. model_00009.weights.h5 +3 -0
  16. model_00010.weights.h5 +3 -0
  17. model_00011.weights.h5 +3 -0
  18. model_00012.weights.h5 +3 -0
  19. model_00013.weights.h5 +3 -0
  20. model_00014.weights.h5 +3 -0
  21. model_00015.weights.h5 +3 -0
  22. model_00016.weights.h5 +3 -0
  23. model_00017.weights.h5 +3 -0
  24. model_00018.weights.h5 +3 -0
  25. model_00019.weights.h5 +3 -0
  26. model_00020.weights.h5 +3 -0
  27. model_00021.weights.h5 +3 -0
  28. model_00022.weights.h5 +3 -0
  29. model_00023.weights.h5 +3 -0
  30. model_00024.weights.h5 +3 -0
  31. model_00025.weights.h5 +3 -0
  32. model_00026.weights.h5 +3 -0
  33. model_00027.weights.h5 +3 -0
  34. tokenizer.json +0 -0
.gitattributes ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: image-text-to-text
3
+ license: other
4
+ license_name: qwen
5
+ license_link: https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/blob/main/LICENSE
6
+ base_model: Qwen/Qwen2.5-VL-72B-Instruct
7
+ library_name: kerasformers
8
+ language:
9
+ - en
10
+ tags:
11
+ - keras
12
+ - kerasformers
13
+ - qwen2_5_vl
14
+ - qwen2.5-vl
15
+ - multimodal
16
+ - vision
17
+ - image-text-to-text
18
+ - pytorch
19
+ - jax
20
+ - tf
21
+ ---
22
+
23
+ *See [our collection](https://huggingface.co/collections/kerasformers/qwen25-vl-6a7cc9f463d6956b6c3ba911) for all Qwen2.5-VL sizes.*
24
+
25
+ # Run Qwen2.5-VL with Keras 3: JAX, PyTorch, or TensorFlow
26
+
27
+ [![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-181717?logo=github)](https://github.com/IMvision12/KerasFormers) [![Docs](https://img.shields.io/badge/Docs-Qwen2.5--VL-1f6feb)](https://imvision12.github.io/KerasFormers/qwen2_5_vl/) [![HuggingFace](https://img.shields.io/badge/HuggingFace-Qwen2.5--VL-ffd21e?logo=huggingface&logoColor=black)](https://huggingface.co/collections/kerasformers/qwen25-vl-6a7cc9f463d6956b6c3ba911)
28
+
29
+ # kerasformers/qwen2.5-vl-72b-instruct
30
+
31
+ Pure-**Keras 3** conversion of [`Qwen/Qwen2.5-VL-72B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**. This is the **72B** variant, the largest Qwen2.5-VL, served here as **image + text -> text** via `Qwen2_5VLProcessor`; weights are sharded and stored in **bfloat16**.
32
+
33
+ For model details, license, and usage terms, see the upstream [model card](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct).
34
+
35
+ Paper: [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution (arXiv:2409.12191)](https://arxiv.org/abs/2409.12191) · [HF Papers](https://huggingface.co/papers/2409.12191)
36
+
37
+ Paper: [YaRN: Efficient Context Window Extension of Large Language Models (arXiv:2309.00071)](https://arxiv.org/abs/2309.00071) · [HF Papers](https://huggingface.co/papers/2309.00071)
38
+
39
+ Paper: [Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities (arXiv:2308.12966)](https://arxiv.org/abs/2308.12966) · [HF Papers](https://huggingface.co/papers/2308.12966)
40
+
41
+ ## ✨ Quick start
42
+
43
+ ### Text-only
44
+
45
+ ```python
46
+ import os
47
+ os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
48
+
49
+ from kerasformers.models.qwen2_5_vl import Qwen2_5VLTextGenerate, Qwen2_5VLProcessor
50
+
51
+ model = Qwen2_5VLTextGenerate.from_weights("kerasformers/qwen2.5-vl-72b-instruct")
52
+ processor = Qwen2_5VLProcessor.from_weights("kerasformers/qwen2.5-vl-72b-instruct")
53
+
54
+ inputs = processor(conversation=[
55
+ {"role": "user", "content": [{"type": "text", "text": "Hello, who are you?"}]}
56
+ ])
57
+ outputs = model.generate(**inputs, max_new_tokens=64)
58
+ print(processor.decode(outputs[0]))
59
+ ```
60
+
61
+ ### Image + text
62
+
63
+ ```python
64
+ import os
65
+ os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
66
+
67
+ from PIL import Image
68
+ from kerasformers.models.qwen2_5_vl import Qwen2_5VLConditionalGenerate, Qwen2_5VLProcessor
69
+
70
+ model = Qwen2_5VLConditionalGenerate.from_weights("kerasformers/qwen2.5-vl-72b-instruct")
71
+ processor = Qwen2_5VLProcessor.from_weights("kerasformers/qwen2.5-vl-72b-instruct")
72
+
73
+ inputs = processor(conversation=[
74
+ {"role": "user", "content": [
75
+ {"type": "image", "image": Image.open("photo.jpg")},
76
+ {"type": "text", "text": "Describe this image in one sentence."},
77
+ ]}
78
+ ])
79
+ outputs = model.generate(**inputs, max_new_tokens=64)
80
+ print(processor.decode(outputs[0]))
81
+ ```
82
+
83
+ ## Loading a 72B checkpoint
84
+
85
+ At **72B** the checkpoint is sharded and does not fit comfortably on a
86
+ single consumer GPU. These flags compose, and each trades a different resource:
87
+
88
+ ```python
89
+ model = Qwen2_5VLConditionalGenerate.from_weights(
90
+ "kerasformers/qwen2.5-vl-72b-instruct",
91
+ quantization="int8", # weight-only int8 on Dense / Embedding (~4x smaller)
92
+ )
93
+ ```
94
+
95
+ Load any Qwen2.5-VL variant the same way with `from_weights("kerasformers/<variant>")`:
96
+
97
+ | Variant | Hub |
98
+ | --- | --- |
99
+ | `qwen2.5-vl-3b-instruct` | [kerasformers/qwen2.5-vl-3b-instruct](https://huggingface.co/kerasformers/qwen2.5-vl-3b-instruct) |
100
+ | `qwen2.5-vl-7b-instruct` | [kerasformers/qwen2.5-vl-7b-instruct](https://huggingface.co/kerasformers/qwen2.5-vl-7b-instruct) |
101
+ | `qwen2.5-vl-32b-instruct` | [kerasformers/qwen2.5-vl-32b-instruct](https://huggingface.co/kerasformers/qwen2.5-vl-32b-instruct) |
102
+ | `qwen2.5-vl-72b-instruct` | [kerasformers/qwen2.5-vl-72b-instruct](https://huggingface.co/kerasformers/qwen2.5-vl-72b-instruct) |
103
+
104
+ ## Special Thanks
105
+
106
+ A huge thank you to the Qwen team at Alibaba for creating and releasing these models.
107
+
108
+ License: Qwen license (see the [upstream license](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/blob/main/LICENSE)).
kf_config.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "library_name": "kerasformers",
3
+ "kerasformers_version": "1.2.2",
4
+ "model_module": "kerasformers.models.qwen2_5_vl",
5
+ "model_class": "Qwen2_5VLConditionalGenerate",
6
+ "variant": "qwen2.5-vl-72b-instruct",
7
+ "weights": "model.weights.json",
8
+ "schema_version": 2,
9
+ "weight_dtype": "bfloat16",
10
+ "model_type": "qwen2_5_vl",
11
+ "text_config": {
12
+ "vocab_size": 152064,
13
+ "embed_dim": 8192,
14
+ "mlp_dim": 29568,
15
+ "num_layers": 80,
16
+ "num_heads": 64,
17
+ "num_kv_heads": 8,
18
+ "norm_eps": 1e-06,
19
+ "rope_theta": 1000000.0,
20
+ "mrope_section": [
21
+ 16,
22
+ 24,
23
+ 24
24
+ ],
25
+ "tie_embeddings": false
26
+ },
27
+ "vision_config": {
28
+ "depth": 32,
29
+ "embed_dim": 1280,
30
+ "mlp_dim": 3456,
31
+ "num_heads": 16,
32
+ "out_dim": 8192,
33
+ "window_size": 112,
34
+ "fullatt_block_indexes": [
35
+ 7,
36
+ 15,
37
+ 23,
38
+ 31
39
+ ],
40
+ "tokens_per_second": 2,
41
+ "patch_size": 14,
42
+ "spatial_merge_size": 2,
43
+ "temporal_patch_size": 2,
44
+ "in_channels": 3
45
+ },
46
+ "image_token_id": 151655,
47
+ "video_token_id": 151656,
48
+ "vision_start_token_id": 151652,
49
+ "vision_end_token_id": 151653
50
+ }
kf_preprocessor.json ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "library_name": "kerasformers",
3
+ "kerasformers_version": "1.2.2",
4
+ "preprocessor_module": "kerasformers.models.qwen2_vl",
5
+ "preprocessor_class": "Qwen2VLImageProcessor",
6
+ "variant": "qwen2.5-vl-72b-instruct",
7
+ "patch_size": 14,
8
+ "spatial_merge_size": 2,
9
+ "temporal_patch_size": 2,
10
+ "min_pixels": 3136,
11
+ "max_pixels": 1003520,
12
+ "image_mean": [
13
+ 0.48145467042922974,
14
+ 0.45782750844955444,
15
+ 0.40821072459220886
16
+ ],
17
+ "image_std": [
18
+ 0.2686295509338379,
19
+ 0.2613025903701782,
20
+ 0.27577710151672363
21
+ ]
22
+ }
model.weights.json ADDED
The diff for this file is too large to render. See raw diff
 
model_00000.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3dc8d4dc9e9a95de3d972dce2f575e3239a4971bd643a910b526761d1851ba54
3
+ size 5361837096
model_00001.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de67f2c4172185e5ca48a1300946ac8659a87d410231930200e2904004dd2aef
3
+ size 5290619432
model_00002.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff92bd3f22ead5b934d1665bd001bedef95ca929958b8c529f94444f6679e5b0
3
+ size 5074629160
model_00003.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b3871afa1d54c70adc3aa67fa29d56b7c117d4440a1ebb091fddc5d57378c1cf
3
+ size 4971825704
model_00004.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c0633bf00be3d42ac3bc56b92cee9ffb707b21da025e8b1c1caec5c35b65ffd5
3
+ size 5273821736
model_00005.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6f616f82cf971bca625731e090775ed9808d1c0e927251201849d1f1e77a8b7
3
+ size 5290635816
model_00006.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5fbbc6a7673ea796ab263982be9eaf3cd9807f60b2bf87bc6a5ab255c0efa7f3
3
+ size 5257042472
model_00007.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:96a3f4498ed77abd01df6d696fdd1e611fd83e46e985c645ebfae501e6e2ae51
3
+ size 5273838120
model_00008.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e5bb029dceb4ded2df181fe0c9ace526bcfa99fd1e20d564a82680ada14155bb
3
+ size 5290635816
model_00009.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9cd3bf921a55a72b4d8093914b5bf75c30c7e80695df2e5c19859947b5c85a15
3
+ size 5257042472
model_00010.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc7c9033bd1de86eb7ed943376bf134d56a956a3c568c632e79c4f49a0dbc2a1
3
+ size 5273838120
model_00011.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cd982d564940b706f8fba19d965187ad6020326c775dd5bb44ba32993bf75323
3
+ size 5290635816
model_00012.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60c2dac25adaeb76e310b0833d433cec3b85856558c31d8b04deec21ef4bb03f
3
+ size 5257042472
model_00013.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1251a27c899697626363097c32d16d38b3ff70032a634cd59446c0d8b94134fd
3
+ size 5273838120
model_00014.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f94f586d7fa9fb7a2f698e4fcac85722c1248da1a1530ccc6a3a217fc3514d8
3
+ size 5290635816
model_00015.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e43d2c7c13ab9914375ae2a6a64d94e4cf1ce76ec3206aaf35d2948ee0db464
3
+ size 5257042472
model_00016.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9ed5d1679276537926ba3f890df892a075c5636cbc1b2127ec7cdb624abfb7ed
3
+ size 5273838120
model_00017.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1593ac7af123922fd2a68454666a8b44dd86ac5c6645260e1c86b1ec7cf27a65
3
+ size 5290635816
model_00018.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d17bcbfdacabc07c8db0d707b7076a2cb8bfc477a5e7ac135f16026cb6c3609
3
+ size 5257042472
model_00019.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2155d97fd65f42297f4fe584a898fb8af4ca8337237fe8d6f9cc5f4f31201512
3
+ size 5273838120
model_00020.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:55ec12d057b13d3672f1e47995a9327f18dc041adf8a701e074b530e10436667
3
+ size 5290635816
model_00021.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bebc21b7186b607221bdb75a61c232b2204a20e813a08136c33b12c31dff5d5f
3
+ size 5257042472
model_00022.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a139420fc499c90c456bc73408d9010b1d6904b15bb48e44e9ece380590a9237
3
+ size 5273838120
model_00023.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8d8248ca352c28f965dc408a07f063822bbab15494881aeddb4917e97640fdb
3
+ size 5290635816
model_00024.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:712508049f2f4b6801a02fe62110d2bba922188e0f29b9811d3e751eb88eac3d
3
+ size 5257042472
model_00025.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e1fcecacbd2a70ab0bacb372b3b4ce296b4a6074d3f463bfe51e30f68f91fb7
3
+ size 5273838120
model_00026.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c14ad4edd65334bd4c9c22b1d4e28e500477d69be7d57c31215bf8dc4d9f12ca
3
+ size 5290635816
model_00027.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:853a3ba93fb93eb21fbc5fdbebe81464fd9a798ccba45baec39a5d4344d914e7
3
+ size 5024197160
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff