Ahmet Yildirim
commited on
Commit
·
dbd1f04
1
Parent(s):
b1bc01c
- Update readme!
Browse files
README.md
CHANGED
|
@@ -49,6 +49,16 @@ from transformers import AutoModel
|
|
| 49 |
humit_tagger = AutoModel.from_pretrained("Humit-Oslo/humit-tagger-xs", trust_remote_code=True)
|
| 50 |
```
|
| 51 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
## Functions and parameters
|
| 53 |
|
| 54 |
The model provides two functions: tag and identify\_language.
|
|
@@ -69,6 +79,7 @@ These functions receive similar parameters.
|
|
| 69 |
| lang\_per\_item | no | yes | True/False | False | consider each item in the list given as separate input for language identification. |
|
| 70 |
| fast\_mode | no | yes | True/False | False | identify languages of the files in the input directory in fast mode. This mode uses only the beginning of the files in identification. This method is much more faster for many files but is not as accurate as if this paramer is set to False. |
|
| 71 |
|
|
|
|
| 72 |
## Several example use cases:
|
| 73 |
|
| 74 |
### Tag one sentence
|
|
|
|
| 49 |
humit_tagger = AutoModel.from_pretrained("Humit-Oslo/humit-tagger-xs", trust_remote_code=True)
|
| 50 |
```
|
| 51 |
|
| 52 |
+
While creating the model, batch\_size and device can be given as parameters
|
| 53 |
+
```python
|
| 54 |
+
humit_tagger = AutoModel.from_pretrained("Humit-Oslo/humit-tagger-xs", trust_remote_code=True, batch_size=16, device="cuda")
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
Here the batch size can be a power of 2, and can be set to higher values, such as 32 or 64, if the model will be loaded on a powerful GPU.
|
| 58 |
+
If the device is not set then the model will try to locate itself on the first CUDA device if it exists, otherwise on CPU.
|
| 59 |
+
A specific device can be set such as "cuda:0" or "cuda:1".
|
| 60 |
+
|
| 61 |
+
|
| 62 |
## Functions and parameters
|
| 63 |
|
| 64 |
The model provides two functions: tag and identify\_language.
|
|
|
|
| 79 |
| lang\_per\_item | no | yes | True/False | False | consider each item in the list given as separate input for language identification. |
|
| 80 |
| fast\_mode | no | yes | True/False | False | identify languages of the files in the input directory in fast mode. This mode uses only the beginning of the files in identification. This method is much more faster for many files but is not as accurate as if this paramer is set to False. |
|
| 81 |
|
| 82 |
+
|
| 83 |
## Several example use cases:
|
| 84 |
|
| 85 |
### Tag one sentence
|