Update README.md
Browse files
README.md
CHANGED
|
@@ -34,6 +34,7 @@ This is the model card of a 🤗 transformers model that has been pushed on the
|
|
| 34 |
- **Demo [optional]:** [More Information Needed]
|
| 35 |
|
| 36 |
## Uses
|
|
|
|
| 37 |
|
| 38 |
from transformers import AutoTokenizer
|
| 39 |
|
|
@@ -42,6 +43,7 @@ shakspear_tokenizer = AutoTokenizer.from_pretrained("otmanheddouch/shakespear-to
|
|
| 42 |
tokens = tokenizer.tokenize(example)
|
| 43 |
tokens_ids = tokenizer.encode(example)
|
| 44 |
|
|
|
|
| 45 |
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 46 |
|
| 47 |
### Direct Use
|
|
|
|
| 34 |
- **Demo [optional]:** [More Information Needed]
|
| 35 |
|
| 36 |
## Uses
|
| 37 |
+
```python
|
| 38 |
|
| 39 |
from transformers import AutoTokenizer
|
| 40 |
|
|
|
|
| 43 |
tokens = tokenizer.tokenize(example)
|
| 44 |
tokens_ids = tokenizer.encode(example)
|
| 45 |
|
| 46 |
+
```
|
| 47 |
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 48 |
|
| 49 |
### Direct Use
|