ANLP Assignment 3 Models: PART1

Model checkpoints and assets for ANLP Assignment 3 (PART1).

Checkpoint Architecture

  • Formulation: Decoder-Only Prefix-LM mapping cipher inputs to plain characters.
  • Architecture: Decoder-Only Transformer (RMSNorm, RoPE, GQA) & Mamba SSM.
  • Tokenization: Character-level tokenizer with fixed 86-token alphabet.

Uploaded Assets

Validation Benchmarks

Model Epoch Val Loss Char Acc Word Acc F1 Levenshtein Params
hybrid 1 1.2204 0.4042 0.3677 0.6137 769.86 1,564,568
transformer 2 1.3912 0.1109 0.0087 0.4814 242.00 2,916,096
mamba 1 1.1309 0.2984 0.2306 0.5411 779.96 1,774,336
gated_deltanet 1 1.1739 0.3165 0.2676 0.5683 772.09 1,354,800

Synchronization & Usage

Download these checkpoints locally using the provided script:

python3 scripts/sync_models.py download --part part1

Or using the shell script:

bash scripts/sync_models.sh download --part part1

Python Loading Example

import torch
from src.models.transformer import DecoderOnlyTransformer

ckpt = torch.load('models/part1/transformer/best_model.pt', map_location='cpu')
config = ckpt['config']
model = DecoderOnlyTransformer(**{k: v for k, v in config.items() if k not in ('model_type', 'gradient_checkpointing')})
model.load_state_dict(ckpt['model_state_dict'])
model.eval()
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support