Update README.md

README.md

@@ -1,32 +1,41 @@
 ---
+license: other
+license_name: climategpt-community-license
+license_link: https://huggingface.co/xquantize/climategpt-7b-4bit-mlx/blob/main/LICENSE.txt
 language:
 - en
-datasets:
-- OpenAssistant/oasst1
-- databricks/databricks-dolly-15k
-base_model: eci-io/climategpt-7b
+library_name: mlx
 tags:
-- climate
 - mlx
-co2_eq_emissions:
-  emissions: 2900
-  training_type: pre-training
-  geographical_location: Washington, USA
-  hardware_used: 8x NVIDIA H100 HBM
-license: other
-license_name: climategpt-community-license
-license_link: https://huggingface.co/eci-io/climategpt-7b/blob/main/LICENSE.txt
-library_name: mlx
+- climate
+- earth-science
+- geoscience
+- quantized
+- 4-bit
+- llama
+base_model: eci-io/climategpt-7b
 pipeline_tag: text-generation
 ---
 
-# xquantize/climategpt-7b-4bit-mlx
+# ClimateGPT-7B-4bit-mlx
+
+This is a 4-bit [MLX](https://github.com/ml-explore/mlx) quantization of [**ClimateGPT-7B**](https://huggingface.co/eci-io/climategpt-7b) (`eci-io/climategpt-7b`), a climate-domain chat model, converted for fast local inference on Apple Silicon.
+
+ClimateGPT is a family of models designed to synthesize interdisciplinary research on climate change. The 7B model is continued-pretrained from Llama-2-7B on a curated collection of climate documents and then instruction fine-tuned on instruction–completion pairs collected in cooperation with climate scientists. The authors report it outperforms Llama-2-70B-Chat on their climate-specific benchmarks. See the original work for full detail:
+
+> Thulke et al., *ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Change*, 2024.
+> Paper: <https://arxiv.org/abs/2401.09646> · Project: <https://eci.io>
+
+## Conversion details
 
-This model [xquantize/climategpt-7b-4bit-mlx](https://huggingface.co/xquantize/climategpt-7b-4bit-mlx) was
-converted to MLX format from [eci-io/climategpt-7b](https://huggingface.co/eci-io/climategpt-7b)
-using mlx-lm version **0.31.3**.
+- **Method:** `mlx_lm.convert` with `-q --q-bits 4`
+- **Result:** 4.500 bits per weight
+- **Precision of base load:** bfloat16
+- **Converted on:** Apple Silicon (MLX)
+- **mlx-lm version:** 0.31.3
+- **Base model:** `eci-io/climategpt-7b` (continued-pretrained from Llama-2-7B)
 
-## Use with mlx
+## Usage
 
 ```bash
 pip install mlx-lm
@@ -37,13 +46,43 @@ from mlx_lm import load, generate
 
 model, tokenizer = load("xquantize/climategpt-7b-4bit-mlx")
 
-prompt = "hello"
+# ClimateGPT is an instruction-tuned CHAT model — use the chat template.
+messages = [
+    {"role": "user", "content": "What is the difference between weather and climate?"}
+]
+prompt = tokenizer.apply_chat_template(
+    messages, tokenize=False, add_generation_prompt=True
+)
+print(generate(model, tokenizer, prompt=prompt, max_tokens=256, verbose=True))
+```
 
-if tokenizer.chat_template is not None:
-    messages = [{"role": "user", "content": prompt}]
-    prompt = tokenizer.apply_chat_template(
-        messages, add_generation_prompt=True, return_dict=False,
-    )
+Or from the command line:
 
-response = generate(model, tokenizer, prompt=prompt, verbose=True)
+```bash
+mlx_lm.generate --model xquantize/climategpt-7b-4bit-mlx \
+  --prompt "What is the difference between weather and climate?" --max-tokens 128
 ```
+
+### Note on usage style
+
+This is an **instruction-tuned chat model**, so prompt it with chat turns via the tokenizer's chat template (shown above) rather than raw text completion. It uses a ChatML-style format (`<|im_start|>` / `<|im_end|>`). The authors note the model is designed to be used with retrieval augmentation (RAG) to extend its knowledge and improve factuality and is **not** intended as a general-purpose chatbot, it is specialized for the climate domain. Context length is 4K tokens.
+
+### Stop-token fix
+
+The upstream `generation_config.json` sets `eos_token_id` to `2` (Llama-2's `</s>`), but this model actually ends turns with the ChatML token `<|im_end|>` (id `32006`). As shipped upstream that mismatch causes generation to run past the end of a turn and repeat. This repository sets `eos_token_id` to `[2, 32006]`, so generation stops cleanly at the end of a turn with no extra flags required.
+
+## Performance
+
+On an Apple Silicon Mac, this 4-bit build runs at ~34 tokens/sec with a peak memory footprint of ~4.0 GB, making a climate-specialized 7B usable on consumer hardware.
+
+## Limitations
+
+ClimateGPT-7B is a 2024-era Llama-2-7B derivative; its general fluency and knowledge reflect that generation, not current frontier models. As with any chat-capable LLM, it may produce biased, offensive or inaccurate responses and should not be relied on for authoritative scientific or safety-critical decisions. It is intended to be paired with retrieval augmentation. Quantization to 4 bits may introduce additional minor quality degradation relative to the full-precision model.
+
+## License & attribution
+
+This model is a 4-bit MLX quantization of ClimateGPT-7B and is distributed under the **ClimateGPT Non-Commercial Community License** (© EQTY R&D, Inc.; Erasmus.AI as third-party beneficiary). **Non-commercial use only.**
+
+Per the license, this repository includes a copy of the license (`LICENSE.txt`) and the required attribution (`Notice`). By using these weights you agree to the ClimateGPT Terms of Service and Acceptable Use Policy contained in `LICENSE.txt`.
+
+All credit for the underlying model belongs to the ClimateGPT authors (Thulke et al.; Erasmus AI, AppTek, EQTY R&D). This repository provides only an MLX-format 4-bit quantization for local inference on Apple Silicon and claims no ownership of the underlying model. The "ClimateGPT" name is used solely to describe the origin of these weights.