Hugging Face's logo

zeromodels
/
deberta_v2_xlarge

Fill-Mask
KerasFormers
Keras
PyTorch
JAX
TensorFlow
deberta
deberta-v2
text-encoder
Model card Files
xet
 Community
IMvision12 commited on
Commit
41ecb7c
·
0 Parent(s):

Super-squash branch 'main' using huggingface_hub

Browse files
Files changed (7) hide show
  1. .gitattributes +35 -0
  2. README.md +101 -0
  3. kf_config.json +31 -0
  4. model.weights.json +606 -0
  5. model_00000.weights.h5 +3 -0
  6. model_00001.weights.h5 +3 -0
  7. tokenizer.json +0 -0
.gitattributes ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: fill-mask
3
+ license: mit
4
+ base_model: microsoft/deberta-v2-xlarge
5
+ library_name: kerasformers
6
+ tags:
7
+ - keras
8
+ - kerasformers
9
+ - deberta
10
+ - deberta-v2
11
+ - fill-mask
12
+ - text-encoder
13
+ - arxiv:2006.03654
14
+ - arxiv:2111.09543
15
+ - pytorch
16
+ - jax
17
+ - tf
18
+ ---
19
+
20
+ ## ***See [our collection](https://huggingface.co/collections/kerasformers/deberta-v1-v2-v3-6a6e90bac01e412b478562f3) for all versions of DeBERTa (v1 / v2 / v3).***
21
+
22
+ # Run DeBERTa with Keras 3: JAX, PyTorch, or TensorFlow
23
+
24
+ [![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-black?logo=github)](https://github.com/IMvision12/KerasFormers) [![Docs](https://img.shields.io/badge/Docs-DeBERTa-blue)](https://imvision12.github.io/KerasFormers/deberta/) [![Collection](https://img.shields.io/badge/HF-DeBERTa%20collection-yellow)](https://huggingface.co/collections/kerasformers/deberta-v1-v2-v3-6a6e90bac01e412b478562f3)
25
+
26
+ # kerasformers/deberta_v2_xlarge
27
+
28
+ Papers: [DeBERTa: Decoding-enhanced BERT with Disentangled Attention (arXiv:2006.03654)](https://arxiv.org/abs/2006.03654) · [DeBERTaV3 (arXiv:2111.09543)](https://arxiv.org/abs/2111.09543) · [HF Papers](https://huggingface.co/papers/2006.03654)
29
+
30
+ DeBERTa is Microsoft's disentangled-attention text encoder (content + relative position). v1 uses byte-level BPE; v2/v3 use SentencePiece. v3 adds ELECTRA-style pretraining with gradient-disentangled embedding sharing. Import from `deberta` / `deberta_v2` / `deberta_v3` to match the generation.
31
+
32
+ For more details on the model, please go to the upstream [model card](https://huggingface.co/microsoft/deberta-v2-xlarge).
33
+
34
+ Pure-**Keras 3** conversion of [`microsoft/deberta-v2-xlarge`](https://huggingface.co/microsoft/deberta-v2-xlarge) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
35
+
36
+ This is a **fill-mask / encoder** checkpoint (`DebertaV2MaskedLM`, v2 xlarge). Task heads (sequence/token classify, QA, …) load via `hf:` fine-tunes.
37
+
38
+ ## ✨ Quick start (fill-mask)
39
+
40
+ ```python
41
+ import os
42
+ os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
43
+
44
+ from kerasformers.models.deberta_v2 import (
45
+ DebertaV2MaskedLM,
46
+ DebertaV2Tokenizer,
47
+ )
48
+
49
+ mlm = DebertaV2MaskedLM.from_weights("kerasformers/deberta_v2_xlarge")
50
+ tokenizer = DebertaV2Tokenizer.from_weights("kerasformers/deberta_v2_xlarge")
51
+
52
+ inputs = tokenizer("The capital of France is [MASK].")
53
+ logits = mlm(inputs) # (1, L, vocab_size)
54
+ mask = int((inputs["input_ids"][0] == tokenizer.mask_token_id).argmax())
55
+ print(tokenizer.decode([int(logits[0, mask].argmax())]))
56
+ ```
57
+
58
+ Load any DeBERTa variant the same way with `from_weights("kerasformers/<variant>")`:
59
+
60
+ | Variant | Hub | Generation |
61
+ |---|---|---|
62
+ | `deberta_base` | [`kerasformers/deberta_base`](https://huggingface.co/kerasformers/deberta_base) | v1 |
63
+ | `deberta_large` | [`kerasformers/deberta_large`](https://huggingface.co/kerasformers/deberta_large) | v1 |
64
+ | `deberta_v2_xlarge` | [`kerasformers/deberta_v2_xlarge`](https://huggingface.co/kerasformers/deberta_v2_xlarge) | v2 |
65
+ | `deberta_v2_xxlarge` | [`kerasformers/deberta_v2_xxlarge`](https://huggingface.co/kerasformers/deberta_v2_xxlarge) | v2 |
66
+ | `deberta_v3_xsmall` | [`kerasformers/deberta_v3_xsmall`](https://huggingface.co/kerasformers/deberta_v3_xsmall) | v3 |
67
+ | `deberta_v3_small` | [`kerasformers/deberta_v3_small`](https://huggingface.co/kerasformers/deberta_v3_small) | v3 |
68
+ | `deberta_v3_base` | [`kerasformers/deberta_v3_base`](https://huggingface.co/kerasformers/deberta_v3_base) | v3 |
69
+ | `deberta_v3_large` | [`kerasformers/deberta_v3_large`](https://huggingface.co/kerasformers/deberta_v3_large) | v3 |
70
+
71
+ ## Available classes
72
+
73
+ Load any of these from this repo with `from_weights("kerasformers/deberta_v2_xlarge")` (or on the fly via the `hf:` prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a `hf:` fine-tune).
74
+
75
+ | Class | Task |
76
+ |---|---|
77
+ | `DebertaV2Model` | Encoder backbone |
78
+ | `DebertaV2MaskedLM` | Masked language modeling (fill-mask) |
79
+ | `DebertaV2SequenceClassify` | Sequence classification |
80
+ | `DebertaV2TokenClassify` | Token classification (NER / POS) |
81
+ | `DebertaV2QnA` | Extractive question answering |
82
+ | `DebertaV2MultipleChoice` | Multiple choice |
83
+
84
+ ```python
85
+ from kerasformers.models.deberta_v2 import DebertaV2SequenceClassify
86
+ model = DebertaV2SequenceClassify.from_weights("kerasformers/deberta_v2_xlarge")
87
+ ```
88
+
89
+ ## Tips
90
+
91
+ - Set `KERAS_BACKEND` **before** importing Keras / kerasformers.
92
+ - Prefer `Tokenizer.from_weights(...)` so vocab and mask token match.
93
+ - Do not mix packages across generations (v1 ≠ v2 ≠ v3).
94
+ - See [DeBERTa docs](https://imvision12.github.io/KerasFormers/deberta/) and [Loading Weights](https://imvision12.github.io/KerasFormers/loading_weights/).
95
+ - Community / upstream safetensors still work via the `hf:` prefix, e.g. `DebertaV2MaskedLM.from_weights("hf:microsoft/deberta-v2-xlarge")`.
96
+
97
+ ## Special Thanks
98
+
99
+ A huge thank you to the Microsoft DeBERTa authors for creating and releasing these models.
100
+
101
+ License: MIT.
kf_config.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "library_name": "kerasformers",
3
+ "kerasformers_version": "1.2.1",
4
+ "model_module": "kerasformers.models.deberta_v2",
5
+ "model_class": "DebertaV2Model",
6
+ "variant": "deberta_v2_xlarge",
7
+ "weights": "model.weights.json",
8
+ "schema_version": 2,
9
+ "weight_dtype": "float32",
10
+ "model_type": "deberta_v2",
11
+ "text_config": {
12
+ "vocab_size": 128100,
13
+ "embed_dim": 1536,
14
+ "num_layers": 24,
15
+ "num_heads": 24,
16
+ "mlp_dim": 6144,
17
+ "max_position_embeddings": 512,
18
+ "max_relative_positions": 512,
19
+ "position_buckets": 256,
20
+ "pos_att_type": [
21
+ "p2c",
22
+ "c2p"
23
+ ],
24
+ "norm_rel_ebd": true,
25
+ "conv_kernel_size": 3,
26
+ "conv_act": "gelu",
27
+ "hidden_act": "gelu",
28
+ "layer_norm_eps": 1e-07,
29
+ "pad_token_id": 0
30
+ }
31
+ }
model.weights.json ADDED
@@ -0,0 +1,606 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "total_size": 4335387024.0
4
+ },
5
+ "weight_map": {
6
+ "/layers/deberta_v2_embeddings/layer_norm/vars": [
7
+ "model_00000.weights.h5"
8
+ ],
9
+ "/layers/deberta_v2_embeddings/word_embeddings/vars": "model_00000.weights.h5",
10
+ "/layers/deberta_v2_relative_embedding/vars": "model_00000.weights.h5",
11
+ "/layers/layer_normalization/vars": [
12
+ "model_00000.weights.h5"
13
+ ],
14
+ "/layers/deberta_v2_disentangled_self_attention/key_proj/vars": [
15
+ "model_00000.weights.h5"
16
+ ],
17
+ "/layers/deberta_v2_disentangled_self_attention/query_proj/vars": [
18
+ "model_00000.weights.h5"
19
+ ],
20
+ "/layers/deberta_v2_disentangled_self_attention/value_proj/vars": [
21
+ "model_00000.weights.h5"
22
+ ],
23
+ "/layers/dense/vars": [
24
+ "model_00000.weights.h5"
25
+ ],
26
+ "/layers/layer_normalization_1/vars": [
27
+ "model_00000.weights.h5"
28
+ ],
29
+ "/layers/dense_1/vars": [
30
+ "model_00000.weights.h5"
31
+ ],
32
+ "/layers/dense_2/vars": [
33
+ "model_00000.weights.h5"
34
+ ],
35
+ "/layers/layer_normalization_2/vars": [
36
+ "model_00000.weights.h5"
37
+ ],
38
+ "/layers/deberta_v2_conv_layer/conv/vars": [
39
+ "model_00000.weights.h5"
40
+ ],
41
+ "/layers/deberta_v2_conv_layer/layer_norm/vars": [
42
+ "model_00000.weights.h5"
43
+ ],
44
+ "/layers/deberta_v2_disentangled_self_attention_1/key_proj/vars": [
45
+ "model_00000.weights.h5"
46
+ ],
47
+ "/layers/deberta_v2_disentangled_self_attention_1/query_proj/vars": [
48
+ "model_00000.weights.h5"
49
+ ],
50
+ "/layers/deberta_v2_disentangled_self_attention_1/value_proj/vars": [
51
+ "model_00000.weights.h5"
52
+ ],
53
+ "/layers/dense_3/vars": [
54
+ "model_00000.weights.h5"
55
+ ],
56
+ "/layers/layer_normalization_3/vars": [
57
+ "model_00000.weights.h5"
58
+ ],
59
+ "/layers/dense_4/vars": [
60
+ "model_00000.weights.h5"
61
+ ],
62
+ "/layers/dense_5/vars": [
63
+ "model_00000.weights.h5"
64
+ ],
65
+ "/layers/layer_normalization_4/vars": [
66
+ "model_00000.weights.h5"
67
+ ],
68
+ "/layers/deberta_v2_disentangled_self_attention_2/key_proj/vars": [
69
+ "model_00000.weights.h5"
70
+ ],
71
+ "/layers/deberta_v2_disentangled_self_attention_2/query_proj/vars": [
72
+ "model_00000.weights.h5"
73
+ ],
74
+ "/layers/deberta_v2_disentangled_self_attention_2/value_proj/vars": [
75
+ "model_00000.weights.h5"
76
+ ],
77
+ "/layers/dense_6/vars": [
78
+ "model_00000.weights.h5"
79
+ ],
80
+ "/layers/layer_normalization_5/vars": [
81
+ "model_00000.weights.h5"
82
+ ],
83
+ "/layers/dense_7/vars": [
84
+ "model_00000.weights.h5"
85
+ ],
86
+ "/layers/dense_8/vars": [
87
+ "model_00000.weights.h5"
88
+ ],
89
+ "/layers/layer_normalization_6/vars": [
90
+ "model_00000.weights.h5"
91
+ ],
92
+ "/layers/deberta_v2_disentangled_self_attention_3/key_proj/vars": [
93
+ "model_00000.weights.h5"
94
+ ],
95
+ "/layers/deberta_v2_disentangled_self_attention_3/query_proj/vars": [
96
+ "model_00000.weights.h5"
97
+ ],
98
+ "/layers/deberta_v2_disentangled_self_attention_3/value_proj/vars": [
99
+ "model_00000.weights.h5"
100
+ ],
101
+ "/layers/dense_9/vars": [
102
+ "model_00000.weights.h5"
103
+ ],
104
+ "/layers/layer_normalization_7/vars": [
105
+ "model_00000.weights.h5"
106
+ ],
107
+ "/layers/dense_10/vars": [
108
+ "model_00000.weights.h5"
109
+ ],
110
+ "/layers/dense_11/vars": [
111
+ "model_00000.weights.h5"
112
+ ],
113
+ "/layers/layer_normalization_8/vars": [
114
+ "model_00000.weights.h5"
115
+ ],
116
+ "/layers/deberta_v2_disentangled_self_attention_4/key_proj/vars": [
117
+ "model_00000.weights.h5"
118
+ ],
119
+ "/layers/deberta_v2_disentangled_self_attention_4/query_proj/vars": [
120
+ "model_00000.weights.h5"
121
+ ],
122
+ "/layers/deberta_v2_disentangled_self_attention_4/value_proj/vars": [
123
+ "model_00000.weights.h5"
124
+ ],
125
+ "/layers/dense_12/vars": [
126
+ "model_00000.weights.h5"
127
+ ],
128
+ "/layers/layer_normalization_9/vars": [
129
+ "model_00000.weights.h5"
130
+ ],
131
+ "/layers/dense_13/vars": [
132
+ "model_00000.weights.h5"
133
+ ],
134
+ "/layers/dense_14/vars": [
135
+ "model_00000.weights.h5"
136
+ ],
137
+ "/layers/layer_normalization_10/vars": [
138
+ "model_00000.weights.h5"
139
+ ],
140
+ "/layers/deberta_v2_disentangled_self_attention_5/key_proj/vars": [
141
+ "model_00000.weights.h5"
142
+ ],
143
+ "/layers/deberta_v2_disentangled_self_attention_5/query_proj/vars": [
144
+ "model_00000.weights.h5"
145
+ ],
146
+ "/layers/deberta_v2_disentangled_self_attention_5/value_proj/vars": [
147
+ "model_00000.weights.h5"
148
+ ],
149
+ "/layers/dense_15/vars": [
150
+ "model_00000.weights.h5"
151
+ ],
152
+ "/layers/layer_normalization_11/vars": [
153
+ "model_00000.weights.h5"
154
+ ],
155
+ "/layers/dense_16/vars": [
156
+ "model_00000.weights.h5"
157
+ ],
158
+ "/layers/dense_17/vars": [
159
+ "model_00000.weights.h5"
160
+ ],
161
+ "/layers/layer_normalization_12/vars": [
162
+ "model_00000.weights.h5"
163
+ ],
164
+ "/layers/deberta_v2_disentangled_self_attention_6/key_proj/vars": [
165
+ "model_00000.weights.h5"
166
+ ],
167
+ "/layers/deberta_v2_disentangled_self_attention_6/query_proj/vars": [
168
+ "model_00000.weights.h5"
169
+ ],
170
+ "/layers/deberta_v2_disentangled_self_attention_6/value_proj/vars": [
171
+ "model_00000.weights.h5"
172
+ ],
173
+ "/layers/dense_18/vars": [
174
+ "model_00000.weights.h5"
175
+ ],
176
+ "/layers/layer_normalization_13/vars": [
177
+ "model_00000.weights.h5"
178
+ ],
179
+ "/layers/dense_19/vars": [
180
+ "model_00000.weights.h5"
181
+ ],
182
+ "/layers/dense_20/vars": [
183
+ "model_00000.weights.h5"
184
+ ],
185
+ "/layers/layer_normalization_14/vars": [
186
+ "model_00000.weights.h5"
187
+ ],
188
+ "/layers/deberta_v2_disentangled_self_attention_7/key_proj/vars": [
189
+ "model_00000.weights.h5"
190
+ ],
191
+ "/layers/deberta_v2_disentangled_self_attention_7/query_proj/vars": [
192
+ "model_00000.weights.h5"
193
+ ],
194
+ "/layers/deberta_v2_disentangled_self_attention_7/value_proj/vars": [
195
+ "model_00000.weights.h5"
196
+ ],
197
+ "/layers/dense_21/vars": [
198
+ "model_00000.weights.h5"
199
+ ],
200
+ "/layers/layer_normalization_15/vars": [
201
+ "model_00000.weights.h5"
202
+ ],
203
+ "/layers/dense_22/vars": [
204
+ "model_00000.weights.h5"
205
+ ],
206
+ "/layers/dense_23/vars": [
207
+ "model_00000.weights.h5"
208
+ ],
209
+ "/layers/layer_normalization_16/vars": [
210
+ "model_00000.weights.h5"
211
+ ],
212
+ "/layers/deberta_v2_disentangled_self_attention_8/key_proj/vars": [
213
+ "model_00000.weights.h5"
214
+ ],
215
+ "/layers/deberta_v2_disentangled_self_attention_8/query_proj/vars": [
216
+ "model_00000.weights.h5"
217
+ ],
218
+ "/layers/deberta_v2_disentangled_self_attention_8/value_proj/vars": [
219
+ "model_00000.weights.h5"
220
+ ],
221
+ "/layers/dense_24/vars": [
222
+ "model_00000.weights.h5"
223
+ ],
224
+ "/layers/layer_normalization_17/vars": [
225
+ "model_00000.weights.h5"
226
+ ],
227
+ "/layers/dense_25/vars": [
228
+ "model_00000.weights.h5"
229
+ ],
230
+ "/layers/dense_26/vars": [
231
+ "model_00000.weights.h5"
232
+ ],
233
+ "/layers/layer_normalization_18/vars": [
234
+ "model_00000.weights.h5"
235
+ ],
236
+ "/layers/deberta_v2_disentangled_self_attention_9/key_proj/vars": [
237
+ "model_00000.weights.h5"
238
+ ],
239
+ "/layers/deberta_v2_disentangled_self_attention_9/query_proj/vars": [
240
+ "model_00000.weights.h5"
241
+ ],
242
+ "/layers/deberta_v2_disentangled_self_attention_9/value_proj/vars": [
243
+ "model_00000.weights.h5"
244
+ ],
245
+ "/layers/dense_27/vars": [
246
+ "model_00000.weights.h5"
247
+ ],
248
+ "/layers/layer_normalization_19/vars": [
249
+ "model_00000.weights.h5"
250
+ ],
251
+ "/layers/dense_28/vars": [
252
+ "model_00000.weights.h5"
253
+ ],
254
+ "/layers/dense_29/vars": [
255
+ "model_00000.weights.h5"
256
+ ],
257
+ "/layers/layer_normalization_20/vars": [
258
+ "model_00000.weights.h5"
259
+ ],
260
+ "/layers/deberta_v2_disentangled_self_attention_10/key_proj/vars": [
261
+ "model_00000.weights.h5"
262
+ ],
263
+ "/layers/deberta_v2_disentangled_self_attention_10/query_proj/vars": [
264
+ "model_00000.weights.h5"
265
+ ],
266
+ "/layers/deberta_v2_disentangled_self_attention_10/value_proj/vars": [
267
+ "model_00000.weights.h5"
268
+ ],
269
+ "/layers/dense_30/vars": [
270
+ "model_00000.weights.h5"
271
+ ],
272
+ "/layers/layer_normalization_21/vars": [
273
+ "model_00000.weights.h5"
274
+ ],
275
+ "/layers/dense_31/vars": [
276
+ "model_00000.weights.h5"
277
+ ],
278
+ "/layers/dense_32/vars": [
279
+ "model_00000.weights.h5"
280
+ ],
281
+ "/layers/layer_normalization_22/vars": [
282
+ "model_00000.weights.h5"
283
+ ],
284
+ "/layers/deberta_v2_disentangled_self_attention_11/key_proj/vars": [
285
+ "model_00000.weights.h5"
286
+ ],
287
+ "/layers/deberta_v2_disentangled_self_attention_11/query_proj/vars": [
288
+ "model_00000.weights.h5"
289
+ ],
290
+ "/layers/deberta_v2_disentangled_self_attention_11/value_proj/vars": [
291
+ "model_00000.weights.h5"
292
+ ],
293
+ "/layers/dense_33/vars": [
294
+ "model_00000.weights.h5"
295
+ ],
296
+ "/layers/layer_normalization_23/vars": [
297
+ "model_00000.weights.h5"
298
+ ],
299
+ "/layers/dense_34/vars": [
300
+ "model_00000.weights.h5"
301
+ ],
302
+ "/layers/dense_35/vars": [
303
+ "model_00000.weights.h5"
304
+ ],
305
+ "/layers/layer_normalization_24/vars": [
306
+ "model_00000.weights.h5"
307
+ ],
308
+ "/layers/deberta_v2_disentangled_self_attention_12/key_proj/vars": [
309
+ "model_00000.weights.h5"
310
+ ],
311
+ "/layers/deberta_v2_disentangled_self_attention_12/query_proj/vars": [
312
+ "model_00000.weights.h5"
313
+ ],
314
+ "/layers/deberta_v2_disentangled_self_attention_12/value_proj/vars": [
315
+ "model_00000.weights.h5"
316
+ ],
317
+ "/layers/dense_36/vars": [
318
+ "model_00000.weights.h5"
319
+ ],
320
+ "/layers/layer_normalization_25/vars": [
321
+ "model_00000.weights.h5"
322
+ ],
323
+ "/layers/dense_37/vars": [
324
+ "model_00000.weights.h5"
325
+ ],
326
+ "/layers/dense_38/vars": [
327
+ "model_00000.weights.h5"
328
+ ],
329
+ "/layers/layer_normalization_26/vars": [
330
+ "model_00000.weights.h5"
331
+ ],
332
+ "/layers/deberta_v2_disentangled_self_attention_13/key_proj/vars": [
333
+ "model_00000.weights.h5"
334
+ ],
335
+ "/layers/deberta_v2_disentangled_self_attention_13/query_proj/vars": [
336
+ "model_00000.weights.h5"
337
+ ],
338
+ "/layers/deberta_v2_disentangled_self_attention_13/value_proj/vars": [
339
+ "model_00000.weights.h5"
340
+ ],
341
+ "/layers/dense_39/vars": [
342
+ "model_00000.weights.h5"
343
+ ],
344
+ "/layers/layer_normalization_27/vars": [
345
+ "model_00000.weights.h5"
346
+ ],
347
+ "/layers/dense_40/vars": [
348
+ "model_00000.weights.h5"
349
+ ],
350
+ "/layers/dense_41/vars": [
351
+ "model_00000.weights.h5"
352
+ ],
353
+ "/layers/layer_normalization_28/vars": [
354
+ "model_00000.weights.h5"
355
+ ],
356
+ "/layers/deberta_v2_disentangled_self_attention_14/key_proj/vars": [
357
+ "model_00000.weights.h5"
358
+ ],
359
+ "/layers/deberta_v2_disentangled_self_attention_14/query_proj/vars": [
360
+ "model_00000.weights.h5"
361
+ ],
362
+ "/layers/deberta_v2_disentangled_self_attention_14/value_proj/vars": [
363
+ "model_00000.weights.h5"
364
+ ],
365
+ "/layers/dense_42/vars": [
366
+ "model_00000.weights.h5"
367
+ ],
368
+ "/layers/layer_normalization_29/vars": [
369
+ "model_00000.weights.h5"
370
+ ],
371
+ "/layers/dense_43/vars": [
372
+ "model_00000.weights.h5"
373
+ ],
374
+ "/layers/dense_44/vars": [
375
+ "model_00000.weights.h5"
376
+ ],
377
+ "/layers/layer_normalization_30/vars": [
378
+ "model_00000.weights.h5"
379
+ ],
380
+ "/layers/deberta_v2_disentangled_self_attention_15/key_proj/vars": [
381
+ "model_00000.weights.h5"
382
+ ],
383
+ "/layers/deberta_v2_disentangled_self_attention_15/query_proj/vars": [
384
+ "model_00000.weights.h5"
385
+ ],
386
+ "/layers/deberta_v2_disentangled_self_attention_15/value_proj/vars": [
387
+ "model_00000.weights.h5"
388
+ ],
389
+ "/layers/dense_45/vars": [
390
+ "model_00000.weights.h5"
391
+ ],
392
+ "/layers/layer_normalization_31/vars": [
393
+ "model_00000.weights.h5"
394
+ ],
395
+ "/layers/dense_46/vars": [
396
+ "model_00000.weights.h5"
397
+ ],
398
+ "/layers/dense_47/vars": [
399
+ "model_00000.weights.h5"
400
+ ],
401
+ "/layers/layer_normalization_32/vars": [
402
+ "model_00000.weights.h5"
403
+ ],
404
+ "/layers/deberta_v2_disentangled_self_attention_16/key_proj/vars": [
405
+ "model_00000.weights.h5"
406
+ ],
407
+ "/layers/deberta_v2_disentangled_self_attention_16/query_proj/vars": [
408
+ "model_00000.weights.h5"
409
+ ],
410
+ "/layers/deberta_v2_disentangled_self_attention_16/value_proj/vars": [
411
+ "model_00000.weights.h5"
412
+ ],
413
+ "/layers/dense_48/vars": [
414
+ "model_00000.weights.h5"
415
+ ],
416
+ "/layers/layer_normalization_33/vars": [
417
+ "model_00000.weights.h5"
418
+ ],
419
+ "/layers/dense_49/vars": [
420
+ "model_00000.weights.h5"
421
+ ],
422
+ "/layers/dense_50/vars": [
423
+ "model_00000.weights.h5"
424
+ ],
425
+ "/layers/layer_normalization_34/vars": [
426
+ "model_00000.weights.h5"
427
+ ],
428
+ "/layers/deberta_v2_disentangled_self_attention_17/key_proj/vars": [
429
+ "model_00000.weights.h5"
430
+ ],
431
+ "/layers/deberta_v2_disentangled_self_attention_17/query_proj/vars": [
432
+ "model_00000.weights.h5"
433
+ ],
434
+ "/layers/deberta_v2_disentangled_self_attention_17/value_proj/vars": [
435
+ "model_00000.weights.h5"
436
+ ],
437
+ "/layers/dense_51/vars": [
438
+ "model_00000.weights.h5"
439
+ ],
440
+ "/layers/layer_normalization_35/vars": [
441
+ "model_00000.weights.h5"
442
+ ],
443
+ "/layers/dense_52/vars": [
444
+ "model_00000.weights.h5"
445
+ ],
446
+ "/layers/dense_53/vars": [
447
+ "model_00000.weights.h5"
448
+ ],
449
+ "/layers/layer_normalization_36/vars": [
450
+ "model_00000.weights.h5"
451
+ ],
452
+ "/layers/deberta_v2_disentangled_self_attention_18/key_proj/vars": [
453
+ "model_00000.weights.h5"
454
+ ],
455
+ "/layers/deberta_v2_disentangled_self_attention_18/query_proj/vars": [
456
+ "model_00000.weights.h5"
457
+ ],
458
+ "/layers/deberta_v2_disentangled_self_attention_18/value_proj/vars": [
459
+ "model_00000.weights.h5"
460
+ ],
461
+ "/layers/dense_54/vars": [
462
+ "model_00000.weights.h5"
463
+ ],
464
+ "/layers/layer_normalization_37/vars": [
465
+ "model_00000.weights.h5"
466
+ ],
467
+ "/layers/dense_55/vars": [
468
+ "model_00000.weights.h5"
469
+ ],
470
+ "/layers/dense_56/vars": [
471
+ "model_00000.weights.h5"
472
+ ],
473
+ "/layers/layer_normalization_38/vars": [
474
+ "model_00000.weights.h5"
475
+ ],
476
+ "/layers/deberta_v2_disentangled_self_attention_19/key_proj/vars": [
477
+ "model_00000.weights.h5"
478
+ ],
479
+ "/layers/deberta_v2_disentangled_self_attention_19/query_proj/vars": [
480
+ "model_00000.weights.h5"
481
+ ],
482
+ "/layers/deberta_v2_disentangled_self_attention_19/value_proj/vars": [
483
+ "model_00000.weights.h5"
484
+ ],
485
+ "/layers/dense_57/vars": [
486
+ "model_00000.weights.h5"
487
+ ],
488
+ "/layers/layer_normalization_39/vars": [
489
+ "model_00000.weights.h5"
490
+ ],
491
+ "/layers/dense_58/vars": [
492
+ "model_00000.weights.h5"
493
+ ],
494
+ "/layers/dense_59/vars": [
495
+ "model_00000.weights.h5"
496
+ ],
497
+ "/layers/layer_normalization_40/vars": [
498
+ "model_00000.weights.h5"
499
+ ],
500
+ "/layers/deberta_v2_disentangled_self_attention_20/key_proj/vars": [
501
+ "model_00000.weights.h5"
502
+ ],
503
+ "/layers/deberta_v2_disentangled_self_attention_20/query_proj/vars": [
504
+ "model_00000.weights.h5"
505
+ ],
506
+ "/layers/deberta_v2_disentangled_self_attention_20/value_proj/vars": [
507
+ "model_00000.weights.h5"
508
+ ],
509
+ "/layers/dense_60/vars": [
510
+ "model_00000.weights.h5"
511
+ ],
512
+ "/layers/layer_normalization_41/vars": [
513
+ "model_00000.weights.h5"
514
+ ],
515
+ "/layers/dense_61/vars": [
516
+ "model_00000.weights.h5"
517
+ ],
518
+ "/layers/dense_62/vars": [
519
+ "model_00000.weights.h5"
520
+ ],
521
+ "/layers/layer_normalization_42/vars": [
522
+ "model_00000.weights.h5"
523
+ ],
524
+ "/layers/deberta_v2_disentangled_self_attention_21/key_proj/vars": [
525
+ "model_00000.weights.h5"
526
+ ],
527
+ "/layers/deberta_v2_disentangled_self_attention_21/query_proj/vars": [
528
+ "model_00000.weights.h5"
529
+ ],
530
+ "/layers/deberta_v2_disentangled_self_attention_21/value_proj/vars": [
531
+ "model_00000.weights.h5"
532
+ ],
533
+ "/layers/dense_63/vars": [
534
+ "model_00000.weights.h5"
535
+ ],
536
+ "/layers/layer_normalization_43/vars": [
537
+ "model_00000.weights.h5"
538
+ ],
539
+ "/layers/dense_64/vars": [
540
+ "model_00000.weights.h5"
541
+ ],
542
+ "/layers/dense_65/vars": [
543
+ "model_00000.weights.h5"
544
+ ],
545
+ "/layers/layer_normalization_44/vars": [
546
+ "model_00000.weights.h5"
547
+ ],
548
+ "/layers/deberta_v2_disentangled_self_attention_22/key_proj/vars": [
549
+ "model_00000.weights.h5"
550
+ ],
551
+ "/layers/deberta_v2_disentangled_self_attention_22/query_proj/vars": [
552
+ "model_00000.weights.h5"
553
+ ],
554
+ "/layers/deberta_v2_disentangled_self_attention_22/value_proj/vars": [
555
+ "model_00000.weights.h5"
556
+ ],
557
+ "/layers/dense_66/vars": [
558
+ "model_00000.weights.h5"
559
+ ],
560
+ "/layers/layer_normalization_45/vars": [
561
+ "model_00000.weights.h5"
562
+ ],
563
+ "/layers/dense_67/vars": [
564
+ "model_00000.weights.h5"
565
+ ],
566
+ "/layers/dense_68/vars": [
567
+ "model_00000.weights.h5"
568
+ ],
569
+ "/layers/layer_normalization_46/vars": [
570
+ "model_00000.weights.h5"
571
+ ],
572
+ "/layers/deberta_v2_disentangled_self_attention_23/key_proj/vars": [
573
+ "model_00000.weights.h5"
574
+ ],
575
+ "/layers/deberta_v2_disentangled_self_attention_23/query_proj/vars": [
576
+ "model_00000.weights.h5"
577
+ ],
578
+ "/layers/deberta_v2_disentangled_self_attention_23/value_proj/vars": [
579
+ "model_00000.weights.h5"
580
+ ],
581
+ "/layers/dense_69/vars": [
582
+ "model_00000.weights.h5"
583
+ ],
584
+ "/layers/layer_normalization_47/vars": [
585
+ "model_00000.weights.h5"
586
+ ],
587
+ "/layers/dense_70/vars": [
588
+ "model_00000.weights.h5"
589
+ ],
590
+ "/layers/dense_71/vars": [
591
+ "model_00000.weights.h5"
592
+ ],
593
+ "/layers/layer_normalization_48/vars": [
594
+ "model_00000.weights.h5"
595
+ ],
596
+ "/layers/dense_72/vars": [
597
+ "model_00000.weights.h5"
598
+ ],
599
+ "/layers/layer_normalization_49/vars": [
600
+ "model_00000.weights.h5"
601
+ ],
602
+ "/layers/dense_73/vars": [
603
+ "model_00001.weights.h5"
604
+ ]
605
+ }
606
+ }
model_00000.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9c8b3fa2f2d45079ef0f39f0c661abab861016a5a7f34d7a72fbfaffc5da32c0
3
+ size 3548741928
model_00001.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1b5ca10a708bd9261107a1eac2356c3e3ea8c5092bc51383989ca2857ac429e
3
+ size 787565016
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff