| language: | |
| - pa | |
| - en | |
| license: other | |
| tags: | |
| - translation | |
| - punjabi | |
| - gurmukhi | |
| - decoder-only | |
| - pytorch | |
| # Gurmukhi SLM Decoder Checkpoint | |
| This repository stores a custom PyTorch decoder-only EN <-> Punjabi Gurmukhi translation checkpoint. | |
| ## Files | |
| - `base_best.pt`: best PyTorch checkpoint with model, optimizer, scheduler, config, and validation loss. | |
| - `decoder_config.json`: extracted model/training config. | |
| - `hf_bpe24k_tokenizer.json`: Hugging Face Tokenizers BPE tokenizer. | |
| ## Validation | |
| Best validation loss from checkpoint: `0.8814973490578788` | |
| ## Loading | |
| Use the project notebook/model class `ModernDecoderOnlyTransformer` and load: | |
| ```python | |
| checkpoint = torch.load("base_best.pt", map_location=device) | |
| model.load_state_dict(checkpoint["model"]) | |
| This is not yet a standard transformers model format. | |