ANLP Assignment 3, Part A: Transformer cipher decoder

802,304 parameters, trained for 20,000 steps. The model reads the ciphertext through a fixed front end: a recurrence removes the key stream, and per-position tables counted on the training split map each symbol to a letter token or to an unknown token. The tables are stored in the sym_table buffer. The network copies known tokens and fills unknown ones from the decoded context; it does not learn the key.

Code and commands: https://code.iiit.ac.in/advanced-nlp/m26-assignment3-vidvathama.babu (bash scripts/part1/test_assn.sh pulls this model and decodes the test split).

Split Exact match (%) Character accuracy (%) Word accuracy (%) Mean Levenshtein
validation 99.57 99.999 99.98 0.010
test 99.47 99.989 99.51 0.096

Earlier stages of the same model are on the branches rule-inputs (rule-candidate inputs) and step-inputs (step and position inputs).

Report

2024122002_a3.pdf in this repository is the submitted report: a 4-page body, then links and references. Code, longer report versions and every result file: https://code.iiit.ac.in/advanced-nlp/m26-assignment3-vidvathama.babu (commit a5ab590).

Downloads last month
95
Safetensors
Model size
2.95M params
Tensor type
F32
ยท
I8
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support