English
jinjieyuan commited on
Commit
d89dade
·
1 Parent(s): e785a97

Create README.md

Browse files

Signed-off-by: jinjieyuan <jinjie.yuan@intel.com>

Files changed (1) hide show
  1. README.md +56 -0
README.md ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ license: apache-2.0
4
+ datasets:
5
+ - nyu-mll/glue
6
+ ---
7
+
8
+ # EFTNAS Model Card: eftnas-s2-bert-medium
9
+
10
+ The super-networks fine-tuned on BERT-medium with [GLUE benchmark](https://gluebenchmark.com/) using EFTNAS.
11
+
12
+ ## Model Details
13
+
14
+ ### Information
15
+
16
+ - **Model name:** eftnas-s2-bert-medium-[TASK]
17
+ - **Base model:** [google/bert_uncased_L-8_H-512_A-8](https://huggingface.co/google/bert_uncased_L-8_H-512_A-8)
18
+ - **Subnetwork version:** Super-network
19
+ - **NNCF Configurations:** [eftnas_configs](https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/EFTNAS/eftnas_configs)
20
+
21
+ ### Training and Evaluation
22
+
23
+ [GLUE benchmark](https://gluebenchmark.com/)
24
+
25
+ ## Results
26
+
27
+ Results of the optimal sub-network discoverd from the super-network:
28
+
29
+ | Model | GFLOPs | GLUE Avg. | MNLI-m | QNLI | QQP | SST-2 | CoLA | MRPC | RTE |
30
+ |-------------------------------|-----------|---------------|----------|------|----------|----------|----------|----------|------|
31
+ | **Test Set:** |
32
+ | [**EFTNAS-S1**]() | 5.7 | 77.7 | 83.7 | 89.9 | 71.8 | 93.4 | 52.6 | 87.6 | 65.0 |
33
+ | [**EFTNAS-S2**]() | 2.2 | 75.2 | 82.0 | 87.8 | 70.6 | 91.4 | 44.5 | 86.1 | 64.0 |
34
+
35
+
36
+ ## Model Sources
37
+
38
+ - **Repository:** [https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/EFTNAS](https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/EFTNAS)
39
+ - **Paper:** [Searching for Efficient Language Models in First-Order Weight-Reordered Super-Networks]()
40
+
41
+ ## Citation
42
+
43
+ ```bibtex
44
+ @inproceedings{
45
+ eftnas2024,
46
+ title={Searching for Efficient Language Models in First-Order Weight-Reordered Super-Networks},
47
+ author={J. Pablo Munoz and Yi Zheng and Nilesh Jain},
48
+ booktitle={The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation},
49
+ year={2024},
50
+ url={}
51
+ }
52
+ ```
53
+
54
+ ## License
55
+
56
+ Apache-2.0