Upload 7 files

README.md

@@ -1,3 +1,83 @@
 ---
-license: mit
+language: sa
+tags:
+- sanskrit
+- bert
+- masked-lm
+- transformers
+license: apache-2.0
+datasets:
+- sanskrit-corpus
+widget:
+- text: "सत्यमेव जयते [MASK]"
+inference: true
 ---
+
+# SanskritBERT (Light)
+
+**SanskritBERT** is a lightweight Transformer model trained specifically for the Sanskrit language. It is based on the BERT architecture and trained using the Masked Language Modeling (MLM) objective.
+
+## Model Description
+
+- **Shared by:** [Your Name / Organization]
+- **Model type:** Transformers Encoder (BERT-like)
+- **Language:** Sanskrit
+- **License:** Apache 2.0
+- **Finetuned from model:** None (Trained from scratch)
+
+### Model Architecture
+- **Layers**: 6
+- **Hidden Size**: 256
+- **Attention Heads**: 4
+- **Feedforward Size**: 1024
+- **Max Sequence Length**: 512
+- **Vocab Size**: 120,000
+- **Parameters**: ~15M
+
+## Intended Uses & Limitations
+
+### Intended Uses
+- Masked Word Prediction
+- Fine-tuning for Sanskrit NLP tasks involves (POS Tagging, NER, Text Classification)
+- Research into low-resource language modeling
+
+### Limitations
+- The model is "Light" (mobile-friendly), so it may not capture as much nuance as a `bert-base` or `bert-large` model.
+- Performance depends heavily on the domain of the downstream task relative to the pre-training corpus.
+
+## Training Data
+
+Trained on a corpus of Sanskrit texts including general literature, wikis, and classical texts.
+
+## Training Procedure
+
+- **Optimizer**: AdamW
+- **Precision**: Mixed Precision (bf16)
+- **Batch Size**: 16
+- **Epochs**: 6
+
+## How to Get Started
+
+You can use the model directly with the Hugging Face `transformers` library:
+
+```python
+from transformers import AutoTokenizer, AutoModelForMaskedLM
+
+tokenizer = AutoTokenizer.from_pretrained("YOUR-USERNAME/SanskritBERT")
+model = AutoModelForMaskedLM.from_pretrained("YOUR-USERNAME/SanskritBERT")
+
+text = "सत्यमेव जयते [MASK]"
+inputs = tokenizer(text, return_tensors="pt")
+outputs = model(**inputs)
+```
+
+## Citation
+
+```bibtex
+@misc{sanskritbert2024,
+  title={SanskritBERT: A Light Transformer Model for Sanskrit},
+  author={[Your Name]},
+  year={2024},
+  publisher={Hugging Face}
+}
+```

config.json

@@ -0,0 +1,24 @@
+{
+  "architectures": [
+    "BertForMaskedLM"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 256,
+  "initializer_range": 0.02,
+  "intermediate_size": 1024,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 4,
+  "num_hidden_layers": 6,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.56.0",
+  "type_vocab_size": 8,
+  "use_cache": true,
+  "vocab_size": 120000
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1886bef8fda23e1c3c346d78a073be916dca561a4933b3fd52186232f76a715f
+size 143126368

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6434bdc90fc1fd3bff989181ee2229b8317a8a217cf51a974e811ae740b816c5
+size 286318795

rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98297750ec38d0e967da7997ba5e9fa0179b503372743eabc305abfeb5669800
+size 14645

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5cdaf34fc8f85feab5c90bce4d80ad546f5f7957e46984713c176e02ab08b5b7
+size 1465

sp_unigram_64k.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93e037cf410c9bf924c6bd7e6b373b624be51f4e4dadde1d9b2a69f2fdf713ac
+size 2071529