---
language: ne
license: cc-by-4.0
base_model: NepBERTa/NepBERTa
tags:
  - nepali
  - bert
  - pytorch-conversion
---

# NepBERTa — PyTorch conversion

A faithful PyTorch conversion of the official
[NepBERTa/NepBERTa](https://huggingface.co/NepBERTa/NepBERTa) checkpoint,
which ships only TensorFlow weights (`tf_model.h5`) that transformers v5
can no longer load.

## Provenance

- Source: `NepBERTa/NepBERTa` @ `tf_model.h5` (TFBertForMaskedLM, 207 tensors).
- Converted 2026-08-04 with `transformers 4.57.6` / `tensorflow-cpu 2.21.0`
  via `load_tf2_checkpoint_in_pytorch_model` into a `BertModel`.
- Tokenizer files copied unmodified from the source repo
  (`vocab.txt` md5 `edfd394677436b306fb062159ec46c72`).
- This repo contains **only the 197 backbone tensors present in the
  official checkpoint** — the source has no trained pooler (it is a
  masked-LM checkpoint), so no pooler weights are shipped; downstream
  loading initializes the pooler freshly, exactly as loading the
  official checkpoint would.
- Cross-check: every converted tensor is bit-identical
  (`torch.equal`) to the independent community port
  [Rajan/nepbertaTorch](https://huggingface.co/Rajan/nepbertaTorch)
  on all 198 tensors that repo shares with the official checkpoint.

## Use

```python
from transformers import AutoModelForSequenceClassification, AutoTokenizer
tok = AutoTokenizer.from_pretrained("subrace/NepBERTa-pytorch")
model = AutoModelForSequenceClassification.from_pretrained(
    "subrace/NepBERTa-pytorch", num_labels=2)
```

All credit for the model itself goes to the NepBERTa authors
([paper](https://aclanthology.org/2022.aacl-short.34/)); this repo exists
only so the weights load in modern PyTorch-only transformers.