Update README.md
Browse files
README.md
CHANGED
|
@@ -4,7 +4,7 @@ GPT3-like T5 model trained to generate text in multiple languages.
|
|
| 4 |
|
| 5 |
## Motivation
|
| 6 |
|
| 7 |
-
- GPT models are expensive run.
|
| 8 |
- GPT models are monolingual.
|
| 9 |
|
| 10 |
## Solution
|
|
@@ -15,7 +15,23 @@ GPT3-like T5 model trained to generate text in multiple languages.
|
|
| 15 |
I fine-tuned T5 on multiple languages (π¬π§ English, π©πͺ German, π«π· French) and multiple academic text snippets from
|
| 16 |
various domains like tech, law, finance and science etc. to generate text, just like GPT models do.
|
| 17 |
|
| 18 |
-
## Usage
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
- Provide some text e.g `"Italy, officially the Italian Republic is a country consisting of"`
|
| 21 |
- Tell Cheapity3 how many words you want to generate e.g `15` -- π Yes, you can control the length.
|
|
@@ -50,17 +66,19 @@ outputs = model.generate(
|
|
| 50 |
|
| 51 |
for i in range(4):
|
| 52 |
print(tokenizer.decode(outputs[i], skip_special_tokens=True, clean_up_tokenization_spaces=True))
|
|
|
|
| 53 |
|
| 54 |
-
|
| 55 |
|
| 56 |
-
|
| 57 |
-
|
|
|
|
| 58 |
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
```
|
| 65 |
|
| 66 |
## Pretty decent right?
|
|
@@ -69,9 +87,9 @@ Hence, whenever you feel like GPT3 is too expensive, Cheapity3 comes to the resc
|
|
| 69 |
|
| 70 |
## Model Training FYI
|
| 71 |
- T5-base model
|
| 72 |
-
- Trained on
|
| 73 |
-
- Mostly text from
|
| 74 |
- Learning rate: 0.00003
|
| 75 |
- 2 epochs
|
| 76 |
- Max input: 512 tokens
|
| 77 |
-
- Max output: 128 tokens
|
|
|
|
| 4 |
|
| 5 |
## Motivation
|
| 6 |
|
| 7 |
+
- GPT models are expensive to run.
|
| 8 |
- GPT models are monolingual.
|
| 9 |
|
| 10 |
## Solution
|
|
|
|
| 15 |
I fine-tuned T5 on multiple languages (π¬π§ English, π©πͺ German, π«π· French) and multiple academic text snippets from
|
| 16 |
various domains like tech, law, finance and science etc. to generate text, just like GPT models do.
|
| 17 |
|
| 18 |
+
## Usage - [NLPlayStore](https://github.com/flexudy/NLPlayStore)π
|
| 19 |
+
|
| 20 |
+
```python
|
| 21 |
+
from store.service_management import ServiceManager
|
| 22 |
+
|
| 23 |
+
service_manager = ServiceManager().get_service("cheapity3")
|
| 24 |
+
|
| 25 |
+
service.install()
|
| 26 |
+
|
| 27 |
+
service = service.launch()
|
| 28 |
+
|
| 29 |
+
input_text = "The mechanical engineering field requires an understanding of core areas including mechanics, dynamics, thermodynamics, materials science, structural analysis, and electricity."
|
| 30 |
+
|
| 31 |
+
generated_texts = service.play(input_text, 15)
|
| 32 |
+
```
|
| 33 |
+
|
| 34 |
+
## Usage - Hugging Face Transformers π€
|
| 35 |
|
| 36 |
- Provide some text e.g `"Italy, officially the Italian Republic is a country consisting of"`
|
| 37 |
- Tell Cheapity3 how many words you want to generate e.g `15` -- π Yes, you can control the length.
|
|
|
|
| 66 |
|
| 67 |
for i in range(4):
|
| 68 |
print(tokenizer.decode(outputs[i], skip_special_tokens=True, clean_up_tokenization_spaces=True))
|
| 69 |
+
```
|
| 70 |
|
| 71 |
+
**INPUT: The mechanical engineering field requires an understanding of core areas including mechanics, dynamics, thermodynamics, materials science, structural analysis, and electricity.**
|
| 72 |
|
| 73 |
+
```
|
| 74 |
+
> Cheapity3 continues with beam search:
|
| 75 |
+
... The field of mechanical engineering is a broad field that includes many core areas of engineering.
|
| 76 |
|
| 77 |
+
> Cheapity3 continues with sampling and top_k=50:
|
| 78 |
+
... Developing the knowledge base for these core areas will enable engineers to build their capabilities rapidly and efficiently. ...
|
| 79 |
+
... The field of mechanics offers a variety and broad range for applications throughout the engineering/technological fields. ...
|
| 80 |
+
... Mechanics generally is not understood by students. While they can be employed in the field, mechanical engineering ...
|
| 81 |
+
... Introduction to mechanical engineering and core fields including chemical products, materials science, structural analysis, and geomatics ...
|
| 82 |
```
|
| 83 |
|
| 84 |
## Pretty decent right?
|
|
|
|
| 87 |
|
| 88 |
## Model Training FYI
|
| 89 |
- T5-base model
|
| 90 |
+
- Trained on ONLY 1M sentences from English, French and German text
|
| 91 |
+
- Mostly text from Wikipedia, arxiv and QA datasets
|
| 92 |
- Learning rate: 0.00003
|
| 93 |
- 2 epochs
|
| 94 |
- Max input: 512 tokens
|
| 95 |
+
- Max output: 128 tokens
|