Upload Gurmukhi decoder base_best checkpoint

README.md

@@ -0,0 +1,35 @@
+---
+language:
+- pa
+- en
+license: other
+tags:
+- translation
+- punjabi
+- gurmukhi
+- decoder-only
+- pytorch
+---
+
+# Gurmukhi SLM Decoder Checkpoint
+
+This repository stores a custom PyTorch decoder-only EN <-> Punjabi Gurmukhi translation checkpoint.
+
+## Files
+
+- `base_best.pt`: best PyTorch checkpoint with model, optimizer, scheduler, config, and validation loss.
+- `decoder_config.json`: extracted model/training config.
+- `hf_bpe24k_tokenizer.json`: Hugging Face Tokenizers BPE tokenizer.
+
+## Validation
+
+Best validation loss from checkpoint: `0.8814973490578788`
+
+## Loading
+
+Use the project notebook/model class `ModernDecoderOnlyTransformer` and load:
+
+```python
+checkpoint = torch.load("base_best.pt", map_location=device)
+model.load_state_dict(checkpoint["model"])
+This is not yet a standard transformers model format.

base_best.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c48ccc523565787833f21878a040a9df1d25e8a7f8ec90adb1f6d11398ec955
+size 550279051

decoder_config.json

@@ -0,0 +1,29 @@
+{
+  "vocab_size": 24000,
+  "pad_id": 0,
+  "eos_id": 3,
+  "profile": "base",
+  "max_seq_len": 256,
+  "min_target_tokens": 48,
+  "d_model": 512,
+  "nhead": 8,
+  "num_layers": 8,
+  "dim_feedforward": 2048,
+  "dropout": 0.1,
+  "rope_base": 10000.0,
+  "gradient_checkpointing": false,
+  "batch_size": 192,
+  "epochs": 4,
+  "lr": 0.0003,
+  "min_lr_ratio": 0.05,
+  "warmup_steps": 2000,
+  "weight_decay": 0.1,
+  "grad_accum_steps": 2,
+  "clip_grad_norm": 1.0,
+  "label_smoothing": 0.05,
+  "val_rows": 10000,
+  "val_fraction": 0.01,
+  "seed": 42,
+  "amp": "bf16",
+  "style_tag": "<natural>"
+}