NBR-1B: Brazilian Portuguese Language Model

NBR-1B is a 1.13 billion parameter language model trained from scratch for Brazilian Portuguese.

Model Details

Attribute	Value
Parameters	1.13B
Architecture	LLaMA-style (GQA, RMSNorm, SwiGLU, RoPE)
Hidden Size	2048
Layers	24
Attention Heads	16
KV Heads	4
Vocabulary	32,000 (BPE)
Context Length	2048
Training Tokens	3.12B
Final Loss	~2.8

Training Data

• CulturaX PT (40%)
• FineWeb-2 PT (52%)
• mC4 PT (5%)
• CC-News PT v2 (2%)
• Books PT (1%)

Usage

This is a base model for text completion. Use with transformers library.

License

Apache 2.0