Commit
·
6eab37f
1
Parent(s):
f0c0638
Update README.md
Browse files
README.md
CHANGED
|
@@ -6,7 +6,12 @@ model-index:
|
|
| 6 |
- name: t5-small-generation-code-documentation
|
| 7 |
results: []
|
| 8 |
widget:
|
| 9 |
-
- text:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
---
|
| 11 |
|
| 12 |
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
|
@@ -58,4 +63,4 @@ The following hyperparameters were used during training:
|
|
| 58 |
- Transformers 4.30.0
|
| 59 |
- Pytorch 2.0.0
|
| 60 |
- Datasets 2.1.0
|
| 61 |
-
- Tokenizers 0.13.3
|
|
|
|
| 6 |
- name: t5-small-generation-code-documentation
|
| 7 |
results: []
|
| 8 |
widget:
|
| 9 |
+
- text: >-
|
| 10 |
+
def get_training_corpus(threshold=256): dataset_corpus = dataset['train']
|
| 11 |
+
for start_idx in range(0, len(dataset_corpus), 1000): samples =
|
| 12 |
+
dataset_corpus[start_idx : start_idx + 1000] samples = [sample for sample in
|
| 13 |
+
samples['func_code_tokens'] if len(sample) < threshold] yield samples
|
| 14 |
+
pipeline_tag: text2text-generation
|
| 15 |
---
|
| 16 |
|
| 17 |
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
|
|
|
| 63 |
- Transformers 4.30.0
|
| 64 |
- Pytorch 2.0.0
|
| 65 |
- Datasets 2.1.0
|
| 66 |
+
- Tokenizers 0.13.3
|