Model card

README.md

@@ -0,0 +1,149 @@
+---
+language:
+  - en
+license: apache-2.0
+base_model: Qwen/Qwen3-8B
+datasets:
+  - HuggingFaceTB/smollm-corpus
+tags:
+  - text-generation
+  - transformers
+  - safetensors
+  - qwen
+  - climate
+  - planetary-boundaries
+  - domain-adaptation
+pipeline_tag: text-generation
+---
+
+# ClimateGPT-3-8B
+
+ClimateGPT-3-8B is an open language model domain-adapted for climate science and the **Planetary Boundaries** framework.
+
+## Model details
+
+- **Base model**: `Qwen/Qwen3-8B`
+- **Model type**: Causal LM
+- **Language(s)**: English
+- **Context length**: 8192 tokens (SFT configuration)
+- **License**: Apache-2.0
+- **Release artifact**: Fully merged weights (standalone model; no adapter required)
+
+## Intended use
+
+- Climate and sustainability Q&A
+- Planetary Boundaries–focused education and analysis
+- Drafting and summarization of climate-related content
+
+## Limitations
+
+- The model may produce incorrect or outdated information.
+- Training data is largely English web content; this can introduce geographic/cultural and topical biases.
+- The model is not a substitute for professional scientific, medical, legal, or policy advice.
+
+## Training
+
+ClimateGPT-3-8B was built in multiple stages:
+
+### Continued pretraining (CPT)
+
+Starting from `Qwen/Qwen3-8B`, we performed continued pretraining on climate-focused corpora primarily derived from FineWeb-Edu (SmolLM-Corpus) using climate- and Planetary Boundaries–oriented filtering.
+
+The data selection emphasizes climate science and Planetary Boundaries terminology and includes filtering to reduce off-topic matches from ambiguous terms.
+
+### Supervised fine-tuning (SFT)
+
+We performed supervised fine-tuning using a mixture of:
+
+- Climate instruction-following data
+- Multi-turn conversations
+- Safety/refusal examples
+- Tool-use data
+- Synthetic climate / Planetary Boundaries Q&A
+
+## Training data
+
+### Public data
+
+- **FineWeb-Edu (via `HuggingFaceTB/smollm-corpus`)**
+  - Used for climate- and Planetary Boundaries–filtered continued pretraining.
+  - **Dataset license**: ODC-By
+  - Dataset page: https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus
+
+### Non-public / generated data
+
+In addition to public data, the training mix includes internal and/or generated instruction data. These datasets are not redistributed with this model.
+
+## Evaluation
+
+We evaluate climate-domain performance using a Planetary Boundaries evaluation suite compatible with EleutherAI’s `lm-evaluation-harness`.
+
+A representative comparison (from this project’s Planetary Boundaries evaluation artifacts) between a ClimateGPT 8B checkpoint and the base Qwen3-8B:
+
+| Task | Metric | ClimateGPT | Qwen3-8B |
+|---|---:|---:|---:|
+| `planetary_boundaries_mcq_large` | acc | 0.4422 | 0.3533 |
+| `planetary_boundaries_mcq_large` | acc_norm | 0.4278 | 0.3900 |
+| `planetary_boundaries_mcq_hard` | acc | 0.3467 | 0.2711 |
+| `planetary_boundaries_mcq_hard` | acc_norm | 0.3800 | 0.3400 |
+| `planetary_boundaries_qa_large` | exact_match | 0.9000 | 0.8467 |
+| `planetary_boundaries_qa_strict_core_nolist` | exact_match | 0.6556 | 0.4889 |
+
+## How to use
+
+### Transformers
+
+This repository contains a standalone model. You can load it directly with Transformers.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "Erasmus-AI/climategpt-3-8b"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
+
+prompt = "Explain the Planetary Boundaries framework in simple terms."
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+out = model.generate(
+    **inputs,
+    max_new_tokens=512,
+    do_sample=True,
+    temperature=0.6,
+    top_p=0.95,
+)
+print(tokenizer.decode(out[0], skip_special_tokens=True))
+```
+
+### vLLM
+
+This model is intended to be compatible with vLLM.
+
+## License
+
+- **Model weights**: Apache-2.0
+- **Base model**: `Qwen/Qwen3-8B` (Apache-2.0)
+
+## Attribution
+
+If you use this model, please cite/attribute the upstream resources where appropriate:
+
+- Base model: https://huggingface.co/Qwen/Qwen3-8B
+- Training data (public portion): https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus (ODC-By)
+
+## Citation
+
+If you use this model in academic work, please cite:
+
+```bibtex
+@misc{climategpt3,
+  title        = {ClimateGPT-3-8B},
+  howpublished = {\url{https://huggingface.co/Erasmus-AI/climategpt-3-8b}},
+  year         = {2026}
+}
+```
+
+## Contact
+
+If you have questions, issues, or evaluation results to share, please open a discussion/issue in the repository that accompanies this release.