Token Classification
Transformers
Safetensors
Arabic
bert
hadith
sanad
matn
hadith-separator
hadith_separator
islam
hadithBERT
Instructions to use SHK4K/hadith-segmentation-bert with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use SHK4K/hadith-segmentation-bert with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="SHK4K/hadith-segmentation-bert")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("SHK4K/hadith-segmentation-bert") model = AutoModelForTokenClassification.from_pretrained("SHK4K/hadith-segmentation-bert", device_map="auto") - Notebooks
- Google Colab
- Kaggle
ismail ismail commited on
Update README.md
Browse files
README.md
CHANGED
|
@@ -52,7 +52,7 @@ model.to(device)
|
|
| 52 |
model.eval()
|
| 53 |
|
| 54 |
# ID mapping dict matching model configuration
|
| 55 |
-
id2label = {0: "
|
| 56 |
|
| 57 |
# 2. Input your raw, unsegmented Hadith text
|
| 58 |
raw_hadith = 'ุญุฏุซูุง ุงูุญู
ูุฏู ุนุจุฏ ุงููู ุจู ุงูุฒุจูุฑ ุ ูุงู : ุญุฏุซูุง ุณููุงู ุ ูุงู : ุญุฏุซูุง ูุญูู ุจู ุณุนูุฏ ุงูุฃูุตุงุฑู ุ ูุงู : ุฃุฎุจุฑูู ู
ุญู
ุฏ ุจู ุฅุจุฑุงููู
ุงูุชูู
ู ุ ุฃูู ุณู
ุน ุนููู
ุฉ ุจู ููุงุต ุงูููุซู ุ ูููู : ุณู
ุนุช ุนู
ุฑ ุจู ุงูุฎุทุงุจ ุฑุถู ุงููู ุนูู ุนูู ุงูู
ูุจุฑุ ูุงู : ุณู
ุนุช ุฑุณูู ุงููู ุตูู ุงููู ุนููู ูุณูู
ุ ูููู : " ุฅูู
ุง ุงูุฃุนู
ุงู ุจุงูููุงุชุ ูุฅูู
ุง ููู ุงู
ุฑุฆ ู
ุง ูููุ ูู
ู ูุงูุช ูุฌุฑุชู ุฅูู ุฏููุง ูุตูุจูุง ุฃู ุฅูู ุงู
ุฑุฃุฉ ูููุญูุงุ ููุฌุฑุชู ุฅูู ู
ุง ูุงุฌุฑ ุฅููู'
|
|
@@ -77,7 +77,6 @@ for token, pred_id in zip(input_tokens, predictions):
|
|
| 77 |
continue
|
| 78 |
|
| 79 |
label = id2label.get(pred_id, "O")
|
| 80 |
-
clean_token = token.replace("##", "") # Handle BERT subword markers
|
| 81 |
|
| 82 |
if "SANAD" in label:
|
| 83 |
sanad_words.append(clean_token)
|
|
|
|
| 52 |
model.eval()
|
| 53 |
|
| 54 |
# ID mapping dict matching model configuration
|
| 55 |
+
id2label = {0: "B-SANAD", 1: "I-SANAD", 2: "B-MATN", 3: "I-MATN"}
|
| 56 |
|
| 57 |
# 2. Input your raw, unsegmented Hadith text
|
| 58 |
raw_hadith = 'ุญุฏุซูุง ุงูุญู
ูุฏู ุนุจุฏ ุงููู ุจู ุงูุฒุจูุฑ ุ ูุงู : ุญุฏุซูุง ุณููุงู ุ ูุงู : ุญุฏุซูุง ูุญูู ุจู ุณุนูุฏ ุงูุฃูุตุงุฑู ุ ูุงู : ุฃุฎุจุฑูู ู
ุญู
ุฏ ุจู ุฅุจุฑุงููู
ุงูุชูู
ู ุ ุฃูู ุณู
ุน ุนููู
ุฉ ุจู ููุงุต ุงูููุซู ุ ูููู : ุณู
ุนุช ุนู
ุฑ ุจู ุงูุฎุทุงุจ ุฑุถู ุงููู ุนูู ุนูู ุงูู
ูุจุฑุ ูุงู : ุณู
ุนุช ุฑุณูู ุงููู ุตูู ุงููู ุนููู ูุณูู
ุ ูููู : " ุฅูู
ุง ุงูุฃุนู
ุงู ุจุงูููุงุชุ ูุฅูู
ุง ููู ุงู
ุฑุฆ ู
ุง ูููุ ูู
ู ูุงูุช ูุฌุฑุชู ุฅูู ุฏููุง ูุตูุจูุง ุฃู ุฅูู ุงู
ุฑุฃุฉ ูููุญูุงุ ููุฌุฑุชู ุฅูู ู
ุง ูุงุฌุฑ ุฅููู'
|
|
|
|
| 77 |
continue
|
| 78 |
|
| 79 |
label = id2label.get(pred_id, "O")
|
|
|
|
| 80 |
|
| 81 |
if "SANAD" in label:
|
| 82 |
sanad_words.append(clean_token)
|