Update README.md
Browse files
README.md
CHANGED
|
@@ -19,6 +19,19 @@ Both regular `.arpa` files as well as more efficient KenLM binary files (`.arpa.
|
|
| 19 |
|
| 20 |
## Usage from within Python
|
| 21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
```python
|
| 23 |
import kenlm
|
| 24 |
from huggingface_hub import hf_hub_download
|
|
|
|
| 19 |
|
| 20 |
## Usage from within Python
|
| 21 |
|
| 22 |
+
Make sure to install dependencies:
|
| 23 |
+
|
| 24 |
+
```shell
|
| 25 |
+
pip install huggingface_hub
|
| 26 |
+
pip install https://github.com/kpu/kenlm/archive/master.zip
|
| 27 |
+
|
| 28 |
+
# If you want to use spaCy preprocessing
|
| 29 |
+
pip install spacy
|
| 30 |
+
python -m spacy download nl_core_news_sm
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
We can then use the Hugging Face hub software to download and cache the model file that we want, and directly use it with KenLM.
|
| 34 |
+
|
| 35 |
```python
|
| 36 |
import kenlm
|
| 37 |
from huggingface_hub import hf_hub_download
|