Hugging Face's logo

bakirgrbic
/
electra-tiny

Text Classification
Transformers
Safetensors
English
electra
fill-mask
Model card Files
xet
Community
bakirgrbic commited on
Commit
052269f
·
verified ·
1 Parent(s): 6853176

Upload v2 model and blimp results

Browse files
Files changed (3) hide show
  1. blimp_results +83 -0
  2. config.json +2 -1
  3. model.safetensors +1 -1
blimp_results ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Running loglikelihood requests: 100%|█████████████████████████████████████████████████████████████| 130186/130186 [1:01:21<00:00, 35.36it/s]
2
+ hf-mlm (pretrained=../../checkpoints/electra-tiny/,backend=mlm), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 1
3
+ | Tasks |Version|Filter|n-shot|Metric|Value | |Stderr|
4
+ |---------------------------------------------------------------------|-------|------|-----:|------|-----:|---|-----:|
5
+ |blimp_supplement |N/A |none | 0|acc |0.4979|± |0.0068|
6
+ | - blimp_supplement_hypernym | 1|none | 0|acc |0.5238|± |0.0172|
7
+ | - blimp_supplement_qa_congruence_easy | 1|none | 0|acc |0.4531|± |0.0627|
8
+ | - blimp_supplement_qa_congruence_tricky | 1|none | 0|acc |0.5818|± |0.0385|
9
+ | - blimp_supplement_subject_aux_inversion | 1|none | 0|acc |0.3949|± |0.0079|
10
+ | - blimp_supplement_turn_taking | 1|none | 0|acc |0.5357|± |0.0299|
11
+ |blimp_filtered |N/A |none | 0|acc |0.5065|± |0.0019|
12
+ | - blimp_adjunct_island_filtered | 1|none | 0|acc |0.4321|± |0.0163|
13
+ | - blimp_anaphor_gender_agreement_filtered | 1|none | 0|acc |0.4830|± |0.0160|
14
+ | - blimp_anaphor_number_agreement_filtered | 1|none | 0|acc |0.6423|± |0.0157|
15
+ | - blimp_animate_subject_passive_filtered | 1|none | 0|acc |0.5140|± |0.0167|
16
+ | - blimp_animate_subject_trans_filtered | 1|none | 0|acc |0.1603|± |0.0121|
17
+ | - blimp_causative_filtered | 1|none | 0|acc |0.4315|± |0.0173|
18
+ | - blimp_complex_NP_island_filtered | 1|none | 0|acc |0.5130|± |0.0172|
19
+ | - blimp_coordinate_structure_constraint_complex_left_branch_filtered| 1|none | 0|acc |0.4570|± |0.0166|
20
+ | - blimp_coordinate_structure_constraint_object_extraction_filtered | 1|none | 0|acc |0.4826|± |0.0162|
21
+ | - blimp_determiner_noun_agreement_1_filtered | 1|none | 0|acc |0.4930|± |0.0164|
22
+ | - blimp_determiner_noun_agreement_2_filtered | 1|none | 0|acc |0.4952|± |0.0164|
23
+ | - blimp_determiner_noun_agreement_irregular_1_filtered | 1|none | 0|acc |0.5477|± |0.0191|
24
+ | - blimp_determiner_noun_agreement_irregular_2_filtered | 1|none | 0|acc |0.4878|± |0.0175|
25
+ | - blimp_determiner_noun_agreement_with_adj_2_filtered | 1|none | 0|acc |0.4708|± |0.0163|
26
+ | - blimp_determiner_noun_agreement_with_adj_irregular_1_filtered | 1|none | 0|acc |0.5432|± |0.0186|
27
+ | - blimp_determiner_noun_agreement_with_adj_irregular_2_filtered | 1|none | 0|acc |0.5976|± |0.0169|
28
+ | - blimp_determiner_noun_agreement_with_adjective_1_filtered | 1|none | 0|acc |0.4845|± |0.0164|
29
+ | - blimp_distractor_agreement_relational_noun_filtered | 1|none | 0|acc |0.5127|± |0.0178|
30
+ | - blimp_distractor_agreement_relative_clause_filtered | 1|none | 0|acc |0.4914|± |0.0169|
31
+ | - blimp_drop_argument_filtered | 1|none | 0|acc |0.5109|± |0.0165|
32
+ | - blimp_ellipsis_n_bar_1_filtered | 1|none | 0|acc |0.4539|± |0.0176|
33
+ | - blimp_ellipsis_n_bar_2_filtered | 1|none | 0|acc |0.3986|± |0.0170|
34
+ | - blimp_existential_there_object_raising_filtered | 1|none | 0|acc |0.5456|± |0.0175|
35
+ | - blimp_existential_there_quantifiers_1_filtered | 1|none | 0|acc |0.4140|± |0.0162|
36
+ | - blimp_existential_there_quantifiers_2_filtered | 1|none | 0|acc |0.4643|± |0.0165|
37
+ | - blimp_existential_there_subject_raising_filtered | 1|none | 0|acc |0.5855|± |0.0162|
38
+ | - blimp_expletive_it_object_raising_filtered | 1|none | 0|acc |0.4822|± |0.0181|
39
+ | - blimp_inchoative_filtered | 1|none | 0|acc |0.5287|± |0.0171|
40
+ | - blimp_intransitive_filtered | 1|none | 0|acc |0.4689|± |0.0169|
41
+ | - blimp_irregular_past_participle_adjectives_filtered | 1|none | 0|acc |0.2799|± |0.0145|
42
+ | - blimp_irregular_past_participle_verbs_filtered | 1|none | 0|acc |0.5255|± |0.0163|
43
+ | - blimp_irregular_plural_subject_verb_agreement_1_filtered | 1|none | 0|acc |0.5100|± |0.0176|
44
+ | - blimp_irregular_plural_subject_verb_agreement_2_filtered | 1|none | 0|acc |0.5415|± |0.0167|
45
+ | - blimp_left_branch_island_echo_question_filtered | 1|none | 0|acc |0.3939|± |0.0159|
46
+ | - blimp_left_branch_island_simple_question_filtered | 1|none | 0|acc |0.5037|± |0.0162|
47
+ | - blimp_matrix_question_npi_licensor_present_filtered | 1|none | 0|acc |0.0000|± |0.0000|
48
+ | - blimp_npi_present_1_filtered | 1|none | 0|acc |0.4356|± |0.0165|
49
+ | - blimp_npi_present_2_filtered | 1|none | 0|acc |0.3950|± |0.0162|
50
+ | - blimp_only_npi_licensor_present_filtered | 1|none | 0|acc |0.4739|± |0.0168|
51
+ | - blimp_only_npi_scope_filtered | 1|none | 0|acc |0.1493|± |0.0123|
52
+ | - blimp_passive_1_filtered | 1|none | 0|acc |0.5024|± |0.0173|
53
+ | - blimp_passive_2_filtered | 1|none | 0|acc |0.5017|± |0.0166|
54
+ | - blimp_principle_A_c_command_filtered | 1|none | 0|acc |0.3446|± |0.0155|
55
+ | - blimp_principle_A_case_1_filtered | 1|none | 0|acc |0.3827|± |0.0161|
56
+ | - blimp_principle_A_case_2_filtered | 1|none | 0|acc |0.5148|± |0.0165|
57
+ | - blimp_principle_A_domain_1_filtered | 1|none | 0|acc |0.3698|± |0.0160|
58
+ | - blimp_principle_A_domain_2_filtered | 1|none | 0|acc |0.5213|± |0.0165|
59
+ | - blimp_principle_A_domain_3_filtered | 1|none | 0|acc |0.4899|± |0.0163|
60
+ | - blimp_principle_A_reconstruction_filtered | 1|none | 0|acc |0.3909|± |0.0157|
61
+ | - blimp_regular_plural_subject_verb_agreement_1_filtered | 1|none | 0|acc |0.5180|± |0.0168|
62
+ | - blimp_regular_plural_subject_verb_agreement_2_filtered | 1|none | 0|acc |0.5196|± |0.0163|
63
+ | - blimp_sentential_negation_npi_licensor_present_filtered | 1|none | 0|acc |1.0000|± |0.0000|
64
+ | - blimp_sentential_negation_npi_scope_filtered | 1|none | 0|acc |0.5281|± |0.0169|
65
+ | - blimp_sentential_subject_island_filtered | 1|none | 0|acc |0.4755|± |0.0161|
66
+ | - blimp_superlative_quantifiers_1_filtered | 1|none | 0|acc |0.6016|± |0.0157|
67
+ | - blimp_superlative_quantifiers_2_filtered | 1|none | 0|acc |0.5213|± |0.0159|
68
+ | - blimp_tough_vs_raising_1_filtered | 1|none | 0|acc |0.4757|± |0.0162|
69
+ | - blimp_tough_vs_raising_2_filtered | 1|none | 0|acc |0.4913|± |0.0165|
70
+ | - blimp_transitive_filtered | 1|none | 0|acc |0.5253|± |0.0170|
71
+ | - blimp_wh_island_filtered | 1|none | 0|acc |0.4896|± |0.0161|
72
+ | - blimp_wh_questions_object_gap_filtered | 1|none | 0|acc |0.9569|± |0.0069|
73
+ | - blimp_wh_questions_subject_gap_filtered | 1|none | 0|acc |0.9944|± |0.0025|
74
+ | - blimp_wh_questions_subject_gap_long_distance_filtered | 1|none | 0|acc |0.9673|± |0.0061|
75
+ | - blimp_wh_vs_that_no_gap_filtered | 1|none | 0|acc |1.0000|± |0.0000|
76
+ | - blimp_wh_vs_that_no_gap_long_distance_filtered | 1|none | 0|acc |0.9954|± |0.0023|
77
+ | - blimp_wh_vs_that_with_gap_filtered | 1|none | 0|acc |0.5484|± |0.0164|
78
+ | - blimp_wh_vs_that_with_gap_long_distance_filtered | 1|none | 0|acc |0.0088|± |0.0031|
79
+
80
+ | Groups |Version|Filter|n-shot|Metric|Value | |Stderr|
81
+ |----------------|-------|------|-----:|------|-----:|---|-----:|
82
+ |blimp_supplement|N/A |none | 0|acc |0.4979|± |0.0068|
83
+ |blimp_filtered |N/A |none | 0|acc |0.5065|± |0.0019|
config.json CHANGED
@@ -1,4 +1,5 @@
1
  {
 
2
  "architectures": [
3
  "ElectraForMaskedLM"
4
  ],
@@ -22,7 +23,7 @@
22
  "summary_type": "first",
23
  "summary_use_proj": true,
24
  "torch_dtype": "float32",
25
- "transformers_version": "4.51.3",
26
  "type_vocab_size": 2,
27
  "use_cache": true,
28
  "vocab_size": 30522
 
1
  {
2
+ "_name_or_path": "electra_owt_full_b256_hs196_ah4_is128_l18_es64_vs30522_pytorch",
3
  "architectures": [
4
  "ElectraForMaskedLM"
5
  ],
 
23
  "summary_type": "first",
24
  "summary_use_proj": true,
25
  "torch_dtype": "float32",
26
+ "transformers_version": "4.49.0",
27
  "type_vocab_size": 2,
28
  "use_cache": true,
29
  "vocab_size": 30522
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bb8ee9170e3178dc882cbf11c3217beb40100a0f220d743c821876715ef16c75
3
  size 23017856
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b0036421435c9a3a0521d95269e4a14d3591c1bc7228b181e18ecc7bc3b49c4f
3
  size 23017856