|
|
| --- |
| license: mit |
| language: en |
| tags: |
| - climate |
| - fill-mask |
| - minilm |
| base_model: microsoft/MiniLM-L12-H384-uncased |
| --- |
| |
| # MiniClimate |
|
|
| A lightweight climate-domain-adapted model based on **MiniLM-L12-H384-uncased** (~33M params), pretrained using DAPT + TAPT on climate-related YouTube comments. |
|
|
| ## What it does |
|
|
| Adapts MiniLM to climate-related text using Masked Language Modelling (MLM). Can be fine-tuned for downstream tasks like: |
| - Stance detection |
| - Environmental claim detection |
| - Sentiment classification |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoModelForMaskedLM, AutoTokenizer |
| |
| tokenizer = AutoTokenizer.from_pretrained("Raj7722/MiniClimate") |
| model = AutoModelForMaskedLM.from_pretrained("Raj7722/MiniClimate") |
| ``` |
|
|
| > This is a pretrained checkpoint. Fine-tune it with a classification head for your downstream task. |
|
|
| ## Results |
|
|
| Avg F1 across 4 downstream tasks (after task-specific fine-tuning): **75.22%** (best F1-per-parameter efficiency vs. RoBERTa, ClimateBERT, SciBERT, DistilBERT). |
|
|
| ## Author |
|
|
| Rajdeep Bose — NSHM Knowledge Campus, Kolkata (M.Sc. Data Science & Analytics) |
|
|
| ## License |
|
|
| MIT |
