distinct
Browse files- distinct.py +5 -4
distinct.py
CHANGED
|
@@ -52,15 +52,16 @@ _CITATION = """\
|
|
| 52 |
|
| 53 |
_DESCRIPTION = """\
|
| 54 |
Distinct metric is to calculate corpus-level diversity of language. We provide two versions of distinct score. Expectation-Adjusted-Distinct (EAD) is the default one, which removes
|
| 55 |
-
the biases of the original distinct score on lengthier sentences. Distinct is the original version.
|
| 56 |
|
| 57 |
-
|
| 58 |
-
<img src="https://huggingface.co/spaces/lsy641/distinct/resolve/main/distinct_compare_pic.jpg" alt="drawing" width="200"/>
|
| 59 |
-
img[alt=drawing]{}
|
| 60 |
|
| 61 |
For the use of Expectation-Adjusted-Distinct, vocab_size is required.
|
| 62 |
|
| 63 |
Please follow ACL paper https://aclanthology.org/2022.acl-short.86 for motivation and follow the rules of thumb provided by https://github.com/lsy641/Expectation-Adjusted-Distinct/blob/main/EAD.ipynb to determine the vocab_size
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
"""
|
| 65 |
|
| 66 |
|
|
|
|
| 52 |
|
| 53 |
_DESCRIPTION = """\
|
| 54 |
Distinct metric is to calculate corpus-level diversity of language. We provide two versions of distinct score. Expectation-Adjusted-Distinct (EAD) is the default one, which removes
|
| 55 |
+
the biases of the original distinct score on lengthier sentences (see Figure below). Distinct is the original version.
|
| 56 |
|
|
|
|
|
|
|
|
|
|
| 57 |
|
| 58 |
For the use of Expectation-Adjusted-Distinct, vocab_size is required.
|
| 59 |
|
| 60 |
Please follow ACL paper https://aclanthology.org/2022.acl-short.86 for motivation and follow the rules of thumb provided by https://github.com/lsy641/Expectation-Adjusted-Distinct/blob/main/EAD.ipynb to determine the vocab_size
|
| 61 |
+
|
| 62 |
+
<img src="https://huggingface.co/spaces/lsy641/distinct/resolve/main/distinct_compare_pic.jpg" alt="drawing" width="200"/>
|
| 63 |
+
img[alt=drawing]{}
|
| 64 |
+
|
| 65 |
"""
|
| 66 |
|
| 67 |
|