Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,47 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
tags:
|
| 5 |
+
- GENIUS
|
| 6 |
+
- conditional text generation
|
| 7 |
+
- sketch-based text generation
|
| 8 |
+
- keywords-to-text generation
|
| 9 |
+
- data augmentation
|
| 10 |
+
license: apache-2.0
|
| 11 |
+
datasets:
|
| 12 |
+
- wikipedia
|
| 13 |
+
widget:
|
| 14 |
+
- text: "machine learning data science my future work"
|
| 15 |
+
example_title: "Example"
|
| 16 |
+
|
| 17 |
+
inference:
|
| 18 |
+
parameters:
|
| 19 |
+
max_length: 100
|
| 20 |
+
num_beams: 2
|
| 21 |
+
do_sample: True
|
| 22 |
+
---
|
| 23 |
+
|
| 24 |
+
# 💡GENIUS – generating text using sketches!
|
| 25 |
+
|
| 26 |
+
- **Paper: [GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation](https://github.com/beyondguo/genius/blob/master/GENIUS_gby_arxiv.pdf)**
|
| 27 |
+
|
| 28 |
+
💡**GENIUS** is a powerful conditional text generation model using sketches as input, which can fill in the missing contexts for a given **sketch** (key information consisting of textual spans, phrases, or words, concatenated by mask tokens). GENIUS is pre-trained on a large- scale textual corpus with a novel *reconstruction from sketch* objective using an *extreme and selective masking* strategy, enabling it to generate diverse and high-quality texts given sketches.
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
- Models hosted in 🤗 Huggingface:
|
| 36 |
+
|
| 37 |
+
**Model variations:**
|
| 38 |
+
|
| 39 |
+
| Model | #params | Language | comment|
|
| 40 |
+
|------------------------|--------------------------------|-------|---------|
|
| 41 |
+
| [`genius-large`](https://huggingface.co/beyond/genius-large) | 406M | English | The version used in **paper** (recommend) |
|
| 42 |
+
| [`genius-large-k2t`](https://huggingface.co/beyond/genius-large-k2t) | 406M | English | keywords-to-text |
|
| 43 |
+
| [`genius-base`](https://huggingface.co/beyond/genius-base) | 139M | English | smaller version |
|
| 44 |
+
| [`genius-base-ps`](https://huggingface.co/beyond/genius-base) | 139M | English | pre-trained both in paragraphs and short sentences |
|
| 45 |
+
| [`genius-base-chinese`](https://huggingface.co/beyond/genius-base-chinese) | 116M | 中文 | 在一千万纯净中文段落上预训练|
|
| 46 |
+
|
| 47 |
+
|