| --- |
| library_name: transformers |
| language: |
| - en |
| license: apache-2.0 |
| base_model: jhu-clsp/mmBERT-base |
| tags: |
| - generated_from_trainer |
| datasets: |
| - SB2 |
| model-index: |
| - name: ReSB2-Base |
| results: [] |
| --- |
| |
| <!-- This model card has been generated automatically according to the information the Trainer had access to. You |
| should probably proofread and complete it, then remove this comment. --> |
|
|
| # ReSB2-Base |
|
|
| ReSB2-Base is a domain-adapted version of [jhu-clsp/mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base), obtained through continued pre-training with the Masked Language Modeling (MLM) objective on the SB2 Dataset. |
|
|
| ## Model description |
|
|
| More information needed |
|
|
| ## Intended uses & limitations |
|
|
| More information needed |
|
|
| ## Training and evaluation data |
|
|
| More information needed |
|
|
| ## Training procedure |
|
|
| ### Training hyperparameters |
|
|
| The following hyperparameters were used during training: |
| - learning_rate: 5e-05 |
| - train_batch_size: 8 |
| - eval_batch_size: 8 |
| - seed: 42 |
| - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments |
| - lr_scheduler_type: linear |
| - num_epochs: 3.0 |
| |
| ### Framework versions |
| |
| - Transformers 4.57.3 |
| - Pytorch 2.13.0+cu130 |
| - Tokenizers 0.22.2 |
| |
| |
| ### Citation |
| |
| If you use this model, please cite: |
| |
| ```bibtex |
| @inproceedings{lage2026resb2, |
| title={ReSB²: Machine-Assisted Linking of Legislative Bills using Domain-Adapted ModernBERT and Explainable AI}, |
| author={Lage, Lucas Gabriel and others}, |
| booktitle={Proceedings of the ACM Conference on Hypertext and Social Media}, |
| year={2026} |
| } |
| ``` |
| |
| |
| ## License |
| |
| The models developed and released in this repository, including the trained ReSB² models, are provided for research and academic purposes. |
| |
| The use of the underlying pre-trained models is subject to the licenses and terms of use defined by their original providers. Users are responsible for complying with the respective licenses when using, modifying, or redistributing these models. |
| |
| |
| ## Acknowledgments |
| |
| This work was supported by the Legislative Assembly of Minas Gerais (ALMG), CNPq, CAPES, and FAPEMIG. |
| |
