Update README.md
Browse files
README.md
CHANGED
|
@@ -4,14 +4,14 @@ language:
|
|
| 4 |
pipeline_tag: text-generation
|
| 5 |
---
|
| 6 |
|
| 7 |
-
# harry-
|
| 8 |
|
| 9 |
-
harry-
|
| 10 |
|
| 11 |
-
- Code and more information in the [GitHub Repository](https://github.com/ShawnLJW/harry-
|
| 12 |
-
- Download the [weights](https://huggingface.co/ShawnLJW/harry-
|
| 13 |
|
| 14 |
-
## Text Generation with harry-
|
| 15 |
> “Ah,” said Mrs. Weasley, hiscolored lips looking unpleasant. “He wasn’t talking about her, he has tried to think he was saying he had looked up. The bleers were flooding.”
|
| 16 |
>
|
| 17 |
> “My master died?” whispered Voldemort, but the wasnoddenbling until he are, making to be seeing him.
|
|
@@ -24,7 +24,7 @@ harry-GPoTter is a transformer text generation model implemented in PyTorch. It
|
|
| 24 |
|
| 25 |
## Model Details
|
| 26 |
|
| 27 |
-
harry-
|
| 28 |
|
| 29 |
The model was trained for 2000 epochs in about 10 minutes with the free tier of Google Colab GPU Runtime. It achieves a cross-entropy loss of 3.1189.
|
| 30 |
|
|
|
|
| 4 |
pipeline_tag: text-generation
|
| 5 |
---
|
| 6 |
|
| 7 |
+
# harry-GPTter
|
| 8 |
|
| 9 |
+
harry-GPTter is a transformer text generation model implemented in PyTorch. It has been trained on text from all 7 books from from all 7 books of the Harry Potter series. In only 10 minutes of training with the free tier of [Google Colaboratory](https://colab.research.google.com/), the model learnt to generate coherent and grammatically correct sentences.
|
| 10 |
|
| 11 |
+
- Code and more information in the [GitHub Repository](https://github.com/ShawnLJW/harry-GPTter)
|
| 12 |
+
- Download the [weights](https://huggingface.co/ShawnLJW/harry-GPTter/resolve/main/checkpoint.pt)
|
| 13 |
|
| 14 |
+
## Text Generation with harry-GPTter
|
| 15 |
> “Ah,” said Mrs. Weasley, hiscolored lips looking unpleasant. “He wasn’t talking about her, he has tried to think he was saying he had looked up. The bleers were flooding.”
|
| 16 |
>
|
| 17 |
> “My master died?” whispered Voldemort, but the wasnoddenbling until he are, making to be seeing him.
|
|
|
|
| 24 |
|
| 25 |
## Model Details
|
| 26 |
|
| 27 |
+
harry-GPTter is a relatively small language model with 56M parameters (less than 1/2x of smallest gpt-2). It contains 8 layers of 8 headed attention with a hidden size of 384. It supports a maximum sequence length of 128. For tokenization, we use the same tokenizer as text-davinci-003, which has a vocabulary of 50,280 in total.
|
| 28 |
|
| 29 |
The model was trained for 2000 epochs in about 10 minutes with the free tier of Google Colab GPU Runtime. It achieves a cross-entropy loss of 3.1189.
|
| 30 |
|