ANLP Assignment 3 Models: PART1
Model checkpoints and assets for ANLP Assignment 3 (PART1).
Checkpoint Architecture
- Formulation: Decoder-Only Prefix-LM mapping cipher inputs to plain characters.
- Architecture: Decoder-Only Transformer (RMSNorm, RoPE, GQA) & Mamba SSM.
- Tokenization: Character-level tokenizer with fixed 86-token alphabet.
Uploaded Assets
Validation Benchmarks
| Model | Epoch | Val Loss | Char Acc | Word Acc | F1 | Levenshtein | Params |
|---|---|---|---|---|---|---|---|
| hybrid | 1 | 1.2204 | 0.4042 | 0.3677 | 0.6137 | 769.86 | 1,564,568 |
| transformer | 2 | 1.3912 | 0.1109 | 0.0087 | 0.4814 | 242.00 | 2,916,096 |
| mamba | 1 | 1.1309 | 0.2984 | 0.2306 | 0.5411 | 779.96 | 1,774,336 |
| gated_deltanet | 1 | 1.1739 | 0.3165 | 0.2676 | 0.5683 | 772.09 | 1,354,800 |
Synchronization & Usage
Download these checkpoints locally using the provided script:
python3 scripts/sync_models.py download --part part1
Or using the shell script:
bash scripts/sync_models.sh download --part part1
Python Loading Example
import torch
from src.models.transformer import DecoderOnlyTransformer
ckpt = torch.load('models/part1/transformer/best_model.pt', map_location='cpu')
config = ckpt['config']
model = DecoderOnlyTransformer(**{k: v for k, v in config.items() if k not in ('model_type', 'gradient_checkpointing')})
model.load_state_dict(ckpt['model_state_dict'])
model.eval()
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support