---
license: apache-2.0
datasets:
- DerivedFunction/sec-filings-snippets-10K
language:
- en
base_model:
- FacebookAI/roberta-base
---

### 📘 Model Description

**FinRoBERTa** is a domain‑adapted variant of **RoBERTa‑base**, trained using **Domain‑Adaptive Pretraining (DAPT)** on the **DerivedFunction/sec-filings-snippets-10K** dataset. This dataset consists of curated excerpts from SEC 10‑K filings, enabling the model to better capture the specialized vocabulary, syntax, and discourse patterns of financial regulatory documents.  

Key characteristics:
- **Base model**: RoBERTa‑base (general‑purpose pretrained transformer)  
- **Adaptation method**: Domain‑Adaptive Pretraining (DAPT)  
- **Domain corpus**: SEC 10‑K filings (snippets)  
- **Language**: English  
- **License**: Apache 2.0  

### 🔍 Intended Use
- As a **foundation for downstream tasks** in financial NLP (e.g., classification, extraction, summarization)  
- Research into domain adaptation techniques and their impact on language model performance  
- Benchmarking DAPT workflows for financial/legal text corpora  

### ⚖️ Limitations
- Not fine‑tuned for specific tasks (classification, QA, summarization) — requires further adaptation for task‑level performance  
- Inherits biases from both the RoBERTa base corpus and SEC filings  
- Not suitable for predictive financial advice or trading decisions