IMvision12 commited on
Commit
4f23b47
·
verified ·
1 Parent(s): aff6ad2

Migrate to zeromodels (rename kf_*.json -> zm_*.json, fix refs in config + README, ensure tag + badge)

Browse files
Files changed (2) hide show
  1. README.md +18 -18
  2. kf_config.json → zm_config.json +23 -23
README.md CHANGED
@@ -2,10 +2,10 @@
2
  pipeline_tag: feature-extraction
3
  license: apache-2.0
4
  base_model: google/electra-small-discriminator
5
- library_name: kerasformers
6
  tags:
7
  - keras
8
- - kerasformers
9
  - electra
10
  - discriminator
11
  - text-encoder
@@ -16,13 +16,13 @@ tags:
16
  - tf
17
  ---
18
 
19
- ## ***See [our collection](https://huggingface.co/collections/kerasformers/electra-6a8540d1f5831e07dc89d8d1) for all versions of ELECTRA.***
20
 
21
  # Run ELECTRA with Keras 3: JAX, PyTorch, or TensorFlow
22
 
23
- [![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-black?logo=github)](https://github.com/IMvision12/KerasFormers) [![Docs](https://img.shields.io/badge/Docs-ELECTRA-blue)](https://imvision12.github.io/KerasFormers/electra/) [![Collection](https://img.shields.io/badge/HF-ELECTRA%20collection-yellow)](https://huggingface.co/collections/kerasformers/electra-6a8540d1f5831e07dc89d8d1)
24
 
25
- # kerasformers/electra_small_discriminator
26
 
27
  Paper: [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators (arXiv:2003.10555)](https://arxiv.org/abs/2003.10555) · [HF Papers](https://huggingface.co/papers/2003.10555)
28
 
@@ -30,7 +30,7 @@ ELECTRA is Google's BERT-style bidirectional text encoder, pre-trained as a repl
30
 
31
  For more details on the model, please go to the upstream [model card](https://huggingface.co/google/electra-small-discriminator).
32
 
33
- Pure-**Keras 3** conversion of [`google/electra-small-discriminator`](https://huggingface.co/google/electra-small-discriminator) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
34
 
35
  ## ✨ Quick start (encoder / downstream)
36
 
@@ -38,27 +38,27 @@ Pure-**Keras 3** conversion of [`google/electra-small-discriminator`](https://hu
38
  import os
39
  os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
40
 
41
- from kerasformers.models.electra import ElectraModel, ElectraTokenizer
42
 
43
- model = ElectraModel.from_weights("kerasformers/electra_small_discriminator")
44
- tokenizer = ElectraTokenizer.from_weights("kerasformers/electra_small_discriminator")
45
 
46
  out = model(tokenizer("The quick brown fox."))["last_hidden_state"] # (1, L, H)
47
  ```
48
 
49
  The same repo also serves the task heads, loaded the same way: `ElectraSequenceClassify`, `ElectraTokenClassify`, `ElectraQnA`, `ElectraMultipleChoice` (each takes the pretrained encoder and a randomly-initialized head, ready for fine-tuning).
50
 
51
- Load any ELECTRA variant the same way with `from_weights("kerasformers/<variant>")`:
52
 
53
  | Size | Discriminator (encoder / downstream) | Generator (masked-LM) |
54
  |---|---|---|
55
- | small | [`kerasformers/electra_small_discriminator`](https://huggingface.co/kerasformers/electra_small_discriminator) | [`kerasformers/electra_small_generator`](https://huggingface.co/kerasformers/electra_small_generator) |
56
- | base | [`kerasformers/electra_base_discriminator`](https://huggingface.co/kerasformers/electra_base_discriminator) | [`kerasformers/electra_base_generator`](https://huggingface.co/kerasformers/electra_base_generator) |
57
- | large | [`kerasformers/electra_large_discriminator`](https://huggingface.co/kerasformers/electra_large_discriminator) | [`kerasformers/electra_large_generator`](https://huggingface.co/kerasformers/electra_large_generator) |
58
 
59
  ## Available classes
60
 
61
- Load any of these from this repo with `from_weights("kerasformers/electra_small_discriminator")` (or on the fly via the `hf:` prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a `hf:` fine-tune).
62
 
63
  | Class | Task |
64
  |---|---|
@@ -69,16 +69,16 @@ Load any of these from this repo with `from_weights("kerasformers/electra_small_
69
  | `ElectraMultipleChoice` | Multiple choice |
70
 
71
  ```python
72
- from kerasformers.models.electra import ElectraSequenceClassify
73
- model = ElectraSequenceClassify.from_weights("kerasformers/electra_small_discriminator")
74
  ```
75
 
76
  ## Tips
77
 
78
- - Set `KERAS_BACKEND` **before** importing Keras / kerasformers.
79
  - Prefer `ElectraTokenizer.from_weights(...)` so WordPiece tokenization matches.
80
  - Downstream tasks (classification / QA / NER) use the **discriminator** repos; the **generator** repos are the masked-LM.
81
- - See [ELECTRA docs](https://imvision12.github.io/KerasFormers/electra/) and [Loading Weights](https://imvision12.github.io/KerasFormers/loading_weights/).
82
  - Community / upstream safetensors still work via the `hf:` prefix, e.g. `ElectraModel.from_weights("hf:google/electra-small-discriminator")`.
83
 
84
  ## Special Thanks
 
2
  pipeline_tag: feature-extraction
3
  license: apache-2.0
4
  base_model: google/electra-small-discriminator
5
+ library_name: zeromodels
6
  tags:
7
  - keras
8
+ - zeromodels
9
  - electra
10
  - discriminator
11
  - text-encoder
 
16
  - tf
17
  ---
18
 
19
+ ## ***See [our collection](https://huggingface.co/collections/zeromodels/electra-6a8540d1f5831e07dc89d8d1) for all versions of ELECTRA.***
20
 
21
  # Run ELECTRA with Keras 3: JAX, PyTorch, or TensorFlow
22
 
23
+ [![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/IMvision12/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-ELECTRA-blue)](https://imvision12.github.io/ZeroModels/electra/) [![Collection](https://img.shields.io/badge/HF-ELECTRA%20collection-yellow)](https://huggingface.co/collections/zeromodels/electra-6a8540d1f5831e07dc89d8d1)
24
 
25
+ # zeromodels/electra_small_discriminator
26
 
27
  Paper: [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators (arXiv:2003.10555)](https://arxiv.org/abs/2003.10555) · [HF Papers](https://huggingface.co/papers/2003.10555)
28
 
 
30
 
31
  For more details on the model, please go to the upstream [model card](https://huggingface.co/google/electra-small-discriminator).
32
 
33
+ Pure-**Keras 3** conversion of [`google/electra-small-discriminator`](https://huggingface.co/google/electra-small-discriminator) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
34
 
35
  ## ✨ Quick start (encoder / downstream)
36
 
 
38
  import os
39
  os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
40
 
41
+ from zeromodels.models.electra import ElectraModel, ElectraTokenizer
42
 
43
+ model = ElectraModel.from_weights("zeromodels/electra_small_discriminator")
44
+ tokenizer = ElectraTokenizer.from_weights("zeromodels/electra_small_discriminator")
45
 
46
  out = model(tokenizer("The quick brown fox."))["last_hidden_state"] # (1, L, H)
47
  ```
48
 
49
  The same repo also serves the task heads, loaded the same way: `ElectraSequenceClassify`, `ElectraTokenClassify`, `ElectraQnA`, `ElectraMultipleChoice` (each takes the pretrained encoder and a randomly-initialized head, ready for fine-tuning).
50
 
51
+ Load any ELECTRA variant the same way with `from_weights("zeromodels/<variant>")`:
52
 
53
  | Size | Discriminator (encoder / downstream) | Generator (masked-LM) |
54
  |---|---|---|
55
+ | small | [`zeromodels/electra_small_discriminator`](https://huggingface.co/zeromodels/electra_small_discriminator) | [`zeromodels/electra_small_generator`](https://huggingface.co/zeromodels/electra_small_generator) |
56
+ | base | [`zeromodels/electra_base_discriminator`](https://huggingface.co/zeromodels/electra_base_discriminator) | [`zeromodels/electra_base_generator`](https://huggingface.co/zeromodels/electra_base_generator) |
57
+ | large | [`zeromodels/electra_large_discriminator`](https://huggingface.co/zeromodels/electra_large_discriminator) | [`zeromodels/electra_large_generator`](https://huggingface.co/zeromodels/electra_large_generator) |
58
 
59
  ## Available classes
60
 
61
+ Load any of these from this repo with `from_weights("zeromodels/electra_small_discriminator")` (or on the fly via the `hf:` prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a `hf:` fine-tune).
62
 
63
  | Class | Task |
64
  |---|---|
 
69
  | `ElectraMultipleChoice` | Multiple choice |
70
 
71
  ```python
72
+ from zeromodels.models.electra import ElectraSequenceClassify
73
+ model = ElectraSequenceClassify.from_weights("zeromodels/electra_small_discriminator")
74
  ```
75
 
76
  ## Tips
77
 
78
+ - Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
79
  - Prefer `ElectraTokenizer.from_weights(...)` so WordPiece tokenization matches.
80
  - Downstream tasks (classification / QA / NER) use the **discriminator** repos; the **generator** repos are the masked-LM.
81
+ - See [ELECTRA docs](https://imvision12.github.io/ZeroModels/electra/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
82
  - Community / upstream safetensors still work via the `hf:` prefix, e.g. `ElectraModel.from_weights("hf:google/electra-small-discriminator")`.
83
 
84
  ## Special Thanks
kf_config.json → zm_config.json RENAMED
@@ -1,24 +1,24 @@
1
- {
2
- "library_name": "kerasformers",
3
- "kerasformers_version": "1.2.4",
4
- "model_module": "kerasformers.models.electra",
5
- "model_class": "ElectraModel",
6
- "variant": "electra_small_discriminator",
7
- "weights": "model.weights.h5",
8
- "schema_version": 2,
9
- "weight_dtype": "float32",
10
- "model_type": "electra",
11
- "text_config": {
12
- "vocab_size": 30522,
13
- "embedding_size": 128,
14
- "embed_dim": 256,
15
- "num_layers": 12,
16
- "num_heads": 4,
17
- "mlp_dim": 1024,
18
- "max_position_embeddings": 512,
19
- "type_vocab_size": 2,
20
- "hidden_act": "gelu",
21
- "layer_norm_eps": 1e-12,
22
- "pad_token_id": 0
23
- }
24
  }
 
1
+ {
2
+ "library_name": "zeromodels",
3
+ "zeromodels_version": "1.2.4",
4
+ "model_module": "zeromodels.models.electra",
5
+ "model_class": "ElectraModel",
6
+ "variant": "electra_small_discriminator",
7
+ "weights": "model.weights.h5",
8
+ "schema_version": 2,
9
+ "weight_dtype": "float32",
10
+ "model_type": "electra",
11
+ "text_config": {
12
+ "vocab_size": 30522,
13
+ "embedding_size": 128,
14
+ "embed_dim": 256,
15
+ "num_layers": 12,
16
+ "num_heads": 4,
17
+ "mlp_dim": 1024,
18
+ "max_position_embeddings": 512,
19
+ "type_vocab_size": 2,
20
+ "hidden_act": "gelu",
21
+ "layer_norm_eps": 1e-12,
22
+ "pad_token_id": 0
23
+ }
24
  }