Commit History
-
fix token_bytes calculation to use raw token bytes instead of round-tripping through str, which corrupts tokens that are not valid standalone UTF-8 (e.g. bytes >= 0x80 became 3-byte U+FFFD). 193/32768 tokens were affected, so bpb of new runs will read ~0.05% higher than old runs. Thank you @ellenjxu #693
Andrej Karpathy +1: Claude Fable 5Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>2ce972a
-
delete the whole report thing i think it was a bad idea it just bloats the code. doing this subtracts 540 LOC
Andrej Karpathyf10bd75
-
fix: correct minor typos in help text, README, and comments
2bitbit2bb93b2
-
quick fix to not OOM main speedrun script
Andrej Karpathy8630d32
-
fix buggy midtrain and update all kwargs to be idiomatic. that is, argparse uses dashes variables use underscores. the underscores are just a remnant of the previous Configurator object. This is the right way
Andrej Karpathy7312ec9
-
nudge hyperparameters of the base script with the results of the sweeps and miniseries. vocab size down to 32K. D:N ratio from 20 to 8. add miniseries script
Andrej Karpathyccf4b7f
-
small change to doc string at top of tok_train.py (#402)
helloaidankUnverified signature389d019
-
initial commit
karpathy3a5e0bc