hafsteinn/icelandic-modernbert-ws-E-xl-s0
Fill-Mask • 1.0B • Updated • 80
Icelandic ModernBERT
Icelandic ModernBERT encoders, 12M to 1B parameters, trained on open data: the size ladder and the warm-start study. MIT licence.
Note Best of the 30 encoders compared in the paper. 1B, warm-started from English Ettin (ettin-encoder-1b), vocabulary transplanted. Warm-start study, arm Ettin, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-S-xl-s0
Fill-Mask • 1.0B • Updated • 58Note 1B, trained from scratch. Warm-start study, arm Scratch, seed 0; main = 80 tokens/param.
hafsteinn/icelandic-modernbert-large-160tpp
Fill-Mask • 0.4B • Updated • 36Note large (378M), trained from scratch on Icelandic, 160 tokens/param (with CulturaX). Size ladder.
hafsteinn/icelandic-modernbert-large
Fill-Mask • 0.4B • Updated • 36Note large (378M), trained from scratch on Icelandic, 80 tokens/param. Size ladder.
hafsteinn/icelandic-modernbert-base-160tpp
Fill-Mask • 0.1B • Updated • 34Note base (136M), trained from scratch on Icelandic, 160 tokens/param (with CulturaX). Size ladder.
hafsteinn/icelandic-modernbert-base
Fill-Mask • 0.1B • Updated • 33Note base (136M), trained from scratch on Icelandic, 80 tokens/param. Size ladder.
hafsteinn/icelandic-modernbert-m-160tpp
Fill-Mask • 59.4M • Updated • 34Note m (59M), trained from scratch on Icelandic, 160 tokens/param (with CulturaX). Size ladder.
hafsteinn/icelandic-modernbert-m
Fill-Mask • 59.4M • Updated • 33Note m (59M), trained from scratch on Icelandic, 80 tokens/param. Size ladder.
hafsteinn/icelandic-modernbert-s-160tpp
Fill-Mask • 25.3M • Updated • 35Note s (25M), trained from scratch on Icelandic, 160 tokens/param (with CulturaX). Size ladder.
hafsteinn/icelandic-modernbert-s
Fill-Mask • 25.3M • Updated • 33Note s (25M), trained from scratch on Icelandic, 80 tokens/param. Size ladder.
hafsteinn/icelandic-modernbert-xs-160tpp
Fill-Mask • 12.4M • Updated • 38Note xs (12M), trained from scratch on Icelandic, 160 tokens/param (with CulturaX). Size ladder.
hafsteinn/icelandic-modernbert-xs
Fill-Mask • 12.4M • Updated • 39Note xs (12M), trained from scratch on Icelandic, 80 tokens/param. Size ladder.
hafsteinn/icelandic-modernbert-ws-E-large-s0
Fill-Mask • 0.4B • Updated • 82Note large (378M), warm-started from English Ettin (ettin-encoder-400m), vocabulary transplanted. Warm-start study, arm Ettin, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-E-base-s0
Fill-Mask • 0.1B • Updated • 80Note base (136M), warm-started from English Ettin (ettin-encoder-150m), vocabulary transplanted. Warm-start study, arm Ettin, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-E-m-s0
Fill-Mask • 59.4M • Updated • 73Note m (59M), warm-started from English Ettin (ettin-encoder-68m), vocabulary transplanted. Warm-start study, arm Ettin, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-E-m-s1
Fill-Mask • 59.4M • Updated • 77Note m (59M), warm-started from English Ettin (ettin-encoder-68m), vocabulary transplanted. Warm-start study, arm Ettin, seed 1; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-E-s-s0
Fill-Mask • 25.3M • Updated • 79Note s (25M), warm-started from English Ettin (ettin-encoder-32m), vocabulary transplanted. Warm-start study, arm Ettin, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-E-s-s1
Fill-Mask • 25.3M • Updated • 76Note s (25M), warm-started from English Ettin (ettin-encoder-32m), vocabulary transplanted. Warm-start study, arm Ettin, seed 1; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-E-xs-s0
Fill-Mask • 12.4M • Updated • 77Note xs (12M), warm-started from English Ettin (ettin-encoder-17m), vocabulary transplanted. Warm-start study, arm Ettin, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-E-xs-s1
Fill-Mask • 12.4M • Updated • 79Note xs (12M), warm-started from English Ettin (ettin-encoder-17m), vocabulary transplanted. Warm-start study, arm Ettin, seed 1; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-Mswap-base-s0
Fill-Mask • 0.1B • Updated • 88Note base (136M), warm-started from mmBERT-base, vocabulary transplanted. Warm-start study, arm mmBERT-transplant, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-Mkeep-base-s0
Fill-Mask • 0.3B • Updated • 78Note base (136M), mmBERT-base continued with its own 256k vocabulary. Warm-start study, arm mmBERT-keep, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-Erand-m-s0
Fill-Mask • 59.4M • Updated • 79Note m (59M), Ettin's layers (ettin-encoder-68m) with new embeddings. Warm-start study, arm Ettin-body, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-Erand-s-s0
Fill-Mask • 25.3M • Updated • 82Note s (25M), Ettin's layers (ettin-encoder-32m) with new embeddings. Warm-start study, arm Ettin-body, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-Erand-xs-s0
Fill-Mask • 12.4M • Updated • 79Note xs (12M), Ettin's layers (ettin-encoder-17m) with new embeddings. Warm-start study, arm Ettin-body, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-S-base-s0
Fill-Mask • 0.1B • Updated • 86Note base (136M), trained from scratch. Warm-start study, arm Scratch, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-S-m-s0
Fill-Mask • 59.4M • Updated • 74Note m (59M), trained from scratch. Warm-start study, arm Scratch, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-S-m-s1
Fill-Mask • 59.4M • Updated • 75Note m (59M), trained from scratch. Warm-start study, arm Scratch, seed 1; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-S-s-s0
Fill-Mask • 25.3M • Updated • 78Note s (25M), trained from scratch. Warm-start study, arm Scratch, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-S-s-s1
Fill-Mask • 25.3M • Updated • 78Note s (25M), trained from scratch. Warm-start study, arm Scratch, seed 1; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-S-xs-s0
Fill-Mask • 12.4M • Updated • 77Note xs (12M), trained from scratch. Warm-start study, arm Scratch, seed 0; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-S-xs-s1
Fill-Mask • 12.4M • Updated • 79Note xs (12M), trained from scratch. Warm-start study, arm Scratch, seed 1; main = 80 tokens/param; branches b10, b20, b40.
hafsteinn/icelandic-modernbert-ws-pilots
Fill-Mask • Updated • 52Note Learning-rate pilots of the warm-start study: the xs pilot and review check C5 (half the peak rate at base), one model per branch. Side experiments, not models to use.