ReSB2-Base

ReSB2-Base is a domain-adapted version of jhu-clsp/mmBERT-base, obtained through continued pre-training with the Masked Language Modeling (MLM) objective on the SB2 Dataset.

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3.0

Framework versions

• Transformers 4.57.3
• Pytorch 2.13.0+cu130
• Tokenizers 0.22.2

Citation

If you use this model, please cite:

@inproceedings{lage2026resb2,
  title={ReSB²: Machine-Assisted Linking of Legislative Bills using Domain-Adapted ModernBERT and Explainable AI},
  author={Lage, Lucas Gabriel and others},
  booktitle={Proceedings of the ACM Conference on Hypertext and Social Media},
  year={2026}
}

License

The models developed and released in this repository, including the trained ReSB² models, are provided for research and academic purposes.

The use of the underlying pre-trained models is subject to the licenses and terms of use defined by their original providers. Users are responsible for complying with the respective licenses when using, modifying, or redistributing these models.

Acknowledgments

This work was supported by the Legislative Assembly of Minas Gerais (ALMG), CNPq, CAPES, and FAPEMIG.