ClimateGPT-3-8B

ClimateGPT-3-8B is an open language model domain-adapted for climate science and the Planetary Boundaries framework.

Model details

• Base model: Qwen/Qwen3-8B
• Model type: Causal LM
• Language(s): English
• Context length: 8192 tokens (SFT configuration)
• License: Apache-2.0
• Release artifact: Fully merged weights (standalone model; no adapter required)

Intended use

• Climate and sustainability Q&A
• Planetary Boundaries–focused education and analysis
• Drafting and summarization of climate-related content

Limitations

• The model may produce incorrect or outdated information.
• Training data is largely English web content; this can introduce geographic/cultural and topical biases.
• The model is not a substitute for professional scientific, medical, legal, or policy advice.

Training

ClimateGPT-3-8B was built in multiple stages:

Continued pretraining (CPT)

Starting from Qwen/Qwen3-8B, we performed continued pretraining on climate-focused corpora primarily derived from FineWeb-Edu (SmolLM-Corpus) using climate- and Planetary Boundaries–oriented filtering.

The data selection emphasizes climate science and Planetary Boundaries terminology and includes filtering to reduce off-topic matches from ambiguous terms.

Supervised fine-tuning (SFT)

We performed supervised fine-tuning using a mixture of:

• Climate instruction-following data
• Multi-turn conversations
• Safety/refusal examples
• Tool-use data
• Synthetic climate / Planetary Boundaries Q&A

Training data

Public data

• FineWeb-Edu (via HuggingFaceTB/smollm-corpus)
  • Used for climate- and Planetary Boundaries–filtered continued pretraining.
  • Dataset license: ODC-By
  • Dataset page: https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus

Non-public / generated data

In addition to public data, the training mix includes internal and/or generated instruction data. These datasets are not redistributed with this model.

Evaluation

We evaluate climate-domain performance using a Planetary Boundaries evaluation suite compatible with EleutherAI’s lm-evaluation-harness.

A representative comparison (from this project’s Planetary Boundaries evaluation artifacts) between a ClimateGPT 8B checkpoint and the base Qwen3-8B:

Task	Metric	ClimateGPT	Qwen3-8B
planetary_boundaries_mcq_large	acc	0.4422	0.3533
planetary_boundaries_mcq_large	acc_norm	0.4278	0.3900
planetary_boundaries_mcq_hard	acc	0.3467	0.2711
planetary_boundaries_mcq_hard	acc_norm	0.3800	0.3400
planetary_boundaries_qa_large	exact_match	0.9000	0.8467
planetary_boundaries_qa_strict_core_nolist	exact_match	0.6556	0.4889

How to use

Transformers

This repository contains a standalone model. You can load it directly with Transformers.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Erasmus-AI/climategpt-3-8b"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

prompt = "Explain the Planetary Boundaries framework in simple terms."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

out = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.6,
    top_p=0.95,
)
print(tokenizer.decode(out[0], skip_special_tokens=True))

vLLM

This model is intended to be compatible with vLLM.

License

• Model weights: Apache-2.0
• Base model: Qwen/Qwen3-8B (Apache-2.0)

Attribution

If you use this model, please cite/attribute the upstream resources where appropriate:

• Base model: https://huggingface.co/Qwen/Qwen3-8B
• Training data (public portion): https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus (ODC-By)

Citation

If you use this model in academic work, please cite:

@misc{climategpt3,
  title        = {ClimateGPT-3-8B},
  howpublished = {\url{https://huggingface.co/Erasmus-AI/climategpt-3-8b}},
  year         = {2026}
}

Contact

If you have questions, issues, or evaluation results to share, please open a discussion/issue in the repository that accompanies this release.