Instructions to use tim1900/bert-chunker with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tim1900/bert-chunker with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="tim1900/bert-chunker")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("tim1900/bert-chunker") model = AutoModel.from_pretrained("tim1900/bert-chunker", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -46,7 +46,7 @@ model.to(device)
|
|
| 46 |
state_dict = safetensors.torch.load_file("./model.safetensors")
|
| 47 |
model.load_state_dict(state_dict)
|
| 48 |
|
| 49 |
-
# text to be chunked
|
| 50 |
text='''In the heart of the bustling city, where towering skyscrapers touch the clouds and the symphony
|
| 51 |
of honking cars never ceases, Sarah, an aspiring novelist, found solace in the quiet corners of the ancient library.
|
| 52 |
Surrounded by shelves that whispered stories of centuries past, she crafted her own world with words, oblivious to the rush outside.
|
|
|
|
| 46 |
state_dict = safetensors.torch.load_file("./model.safetensors")
|
| 47 |
model.load_state_dict(state_dict)
|
| 48 |
|
| 49 |
+
# text to be chunked, no limit on its length
|
| 50 |
text='''In the heart of the bustling city, where towering skyscrapers touch the clouds and the symphony
|
| 51 |
of honking cars never ceases, Sarah, an aspiring novelist, found solace in the quiet corners of the ancient library.
|
| 52 |
Surrounded by shelves that whispered stories of centuries past, she crafted her own world with words, oblivious to the rush outside.
|