---
library_name: transformers
language:
- en
license: apache-2.0
base_model: jhu-clsp/mmBERT-base
tags:
- generated_from_trainer
datasets:
- SB2
model-index:
- name: ReSB2-Base
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# ReSB2-Base

ReSB2-Base is a domain-adapted version of [jhu-clsp/mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base), obtained through continued pre-training with the Masked Language Modeling (MLM) objective on the SB2 Dataset.

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 3.0

### Framework versions

- Transformers 4.57.3
- Pytorch 2.13.0+cu130
- Tokenizers 0.22.2


### Citation

If you use this model, please cite:

```bibtex
@inproceedings{lage2026resb2,
  title={ReSB²: Machine-Assisted Linking of Legislative Bills using Domain-Adapted ModernBERT and Explainable AI},
  author={Lage, Lucas Gabriel and others},
  booktitle={Proceedings of the ACM Conference on Hypertext and Social Media},
  year={2026}
}
```


## License

The models developed and released in this repository, including the trained ReSB² models, are provided for research and academic purposes.

The use of the underlying pre-trained models is subject to the licenses and terms of use defined by their original providers. Users are responsible for complying with the respective licenses when using, modifying, or redistributing these models.


## Acknowledgments

This work was supported by the Legislative Assembly of Minas Gerais (ALMG), CNPq, CAPES, and FAPEMIG.