---
language:
  - en
license: apache-2.0
base_model: Qwen/Qwen3-8B
datasets:
  - HuggingFaceTB/smollm-corpus
tags:
  - text-generation
  - transformers
  - safetensors
  - qwen
  - climate
  - planetary-boundaries
  - domain-adaptation
pipeline_tag: text-generation
---

# ClimateGPT-3-8B

ClimateGPT-3-8B is an open language model domain-adapted for climate science and the **Planetary Boundaries** framework.

## Model details

- **Base model**: `Qwen/Qwen3-8B`
- **Model type**: Causal LM
- **Language(s)**: English
- **Context length**: 8192 tokens (SFT configuration)
- **License**: Apache-2.0
- **Release artifact**: Fully merged weights (standalone model; no adapter required)

## Intended use

- Climate and sustainability Q&A
- Planetary Boundaries–focused education and analysis
- Drafting and summarization of climate-related content

## Limitations

- The model may produce incorrect or outdated information.
- Training data is largely English web content; this can introduce geographic/cultural and topical biases.
- The model is not a substitute for professional scientific, medical, legal, or policy advice.

## Training

ClimateGPT-3-8B was built in multiple stages:

### Continued pretraining (CPT)

Starting from `Qwen/Qwen3-8B`, we performed continued pretraining on climate-focused corpora primarily derived from FineWeb-Edu (SmolLM-Corpus) using climate- and Planetary Boundaries–oriented filtering.

The data selection emphasizes climate science and Planetary Boundaries terminology and includes filtering to reduce off-topic matches from ambiguous terms.

### Supervised fine-tuning (SFT)

We performed supervised fine-tuning using a mixture of:

- Climate instruction-following data
- Multi-turn conversations
- Safety/refusal examples
- Tool-use data
- Synthetic climate / Planetary Boundaries Q&A

## Training data

### Public data

- **FineWeb-Edu (via `HuggingFaceTB/smollm-corpus`)**
  - Used for climate- and Planetary Boundaries–filtered continued pretraining.
  - **Dataset license**: ODC-By
  - Dataset page: https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus

### Non-public / generated data

In addition to public data, the training mix includes internal and/or generated instruction data. These datasets are not redistributed with this model.

## Evaluation

We evaluate climate-domain performance using a Planetary Boundaries evaluation suite compatible with EleutherAI’s `lm-evaluation-harness`.

A representative comparison (from this project’s Planetary Boundaries evaluation artifacts) between a ClimateGPT 8B checkpoint and the base Qwen3-8B:

| Task | Metric | ClimateGPT | Qwen3-8B |
|---|---:|---:|---:|
| `planetary_boundaries_mcq_large` | acc | 0.4422 | 0.3533 |
| `planetary_boundaries_mcq_large` | acc_norm | 0.4278 | 0.3900 |
| `planetary_boundaries_mcq_hard` | acc | 0.3467 | 0.2711 |
| `planetary_boundaries_mcq_hard` | acc_norm | 0.3800 | 0.3400 |
| `planetary_boundaries_qa_large` | exact_match | 0.9000 | 0.8467 |
| `planetary_boundaries_qa_strict_core_nolist` | exact_match | 0.6556 | 0.4889 |

## How to use

### Transformers

This repository contains a standalone model. You can load it directly with Transformers.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Erasmus-AI/climategpt-3-8b"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

prompt = "Explain the Planetary Boundaries framework in simple terms."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

out = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.6,
    top_p=0.95,
)
print(tokenizer.decode(out[0], skip_special_tokens=True))
```

### vLLM

This model is intended to be compatible with vLLM.

## License

- **Model weights**: Apache-2.0
- **Base model**: `Qwen/Qwen3-8B` (Apache-2.0)

## Attribution

If you use this model, please cite/attribute the upstream resources where appropriate:

- Base model: https://huggingface.co/Qwen/Qwen3-8B
- Training data (public portion): https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus (ODC-By)

## Citation

If you use this model in academic work, please cite:

```bibtex
@misc{climategpt3,
  title        = {ClimateGPT-3-8B},
  howpublished = {\url{https://huggingface.co/Erasmus-AI/climategpt-3-8b}},
  year         = {2026}
}
```

## Contact

If you have questions, issues, or evaluation results to share, please open a discussion/issue in the repository that accompanies this release.