Create README.md

README.md

@@ -0,0 +1,97 @@
+---
+language:
+- zh
+- en
+base_model:
+- Qwen/Qwen2.5-72B
+pipeline_tag: text-generation
+library_name: transformers
+---
+## Introduction
+
+The Ming large language model (Ming‑LLM) is a domain‑specialized LLM for the energy sector. 
+- We release both the base model and the supervised fine‑tuned (SFT) variant. 
+- The Ming base model is initialized from the Qwen2.5‑72B base model and is subsequently adapted via continued pretraining on a high‑quality energy‑domain corpus. 
+- The SFT variant is initialized from the Ming base model and is trained on instruction‑tuning datasets, including conversational QA, sentiment analysis, and information extraction, among others.
+- Both models demonstrate improved performance across the C‑Eval, CMMLU, MMLU, GSM8K, and IFEval benchmarks.
+
+## Model Parameters
+Base model:
+- sequence_len: 4096
+- gradient_accumulation_steps: 128
+- learning_rate: 1.0e-5
+- lr_scheduler_type: cosine
+- warmup_ratio: 0
+- num_train_epochs: 1.0
+
+SFT:
+- sequence_len: 4096
+- gradient_accumulation_steps: 128
+- max learning rate: 2e-6
+- max_grad_norm: 1.0
+- lr_scheduler_type: cosine
+- warmup_ratio: 0.03
+- num_train_epochs: 1.0
+
+## Evaluation
+| Model                  | c-eval 5-shot | cmmlu 5-shot | mmlu 5-shot | GPQA 0-shot | BBH 0-shot | HellaSwag 10-shot | GSM8K | IFEVAL |
+|------------------------|---------------|--------------|-------------|-------------|------------|-------------------|-------|--------|
+| qwen2.5-72B-base       | 89.72         | 89.75        | 84.79       | 37.88       | 85.81      | 94.93            | 89.99 | -      |
+| ming1.0-base           | 90.11         | 89.84        | 84.97       | 41.92       | 84.80      | 92.73            | 89.23 | -      |
+| qwen2.5-72B-instruct  | 87.97         | 87.26        | 84.18       | 36.87       | 83.68      | 92.65            | 89.69 | 82.81  |
+| ming1.0              | 90.08         | 89.94        | 85.12       | 37.88       | 85.24      | 94.20            | 91.43 | 78.74  |
+
+## Inference
+
+You can use Ming model with the standard HuggingFace transformers library:
+``` python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM, TextStreamer
+
+dtype = torch.bfloat16
+device_map = "auto"
+
+model_path = /model/path
+tokenizer = AutoTokenizer.from_pretrained(
+    model_path, use_fast=True, trust_remote_code=True
+)
+model = AutoModelForCausalLM.from_pretrained(
+    model_path, torch_dtype=dtype, device_map=device_map, trust_remote_code=True
+)
+
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user",   "content": "who are you?"} 
+]
+
+prompt = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+with torch.no_grad():
+    output_ids = model.generate(
+        **inputs,
+        max_new_tokens=256,
+        do_sample=True,
+        temperature=0.3,
+        top_p=0.9,
+        repetition_penalty=1.1,
+        eos_token_id=eos_token_id,  
+        pad_token_id=(tokenizer.pad_token_id or tokenizer.eos_token_id),
+        streamer=None 
+    )
+gen_ids = output_ids[0, inputs["input_ids"].shape[1]:]
+text = tokenizer.decode(gen_ids, skip_special_tokens=False)
+```
+## Bias, Risks, and Limitations
+- Like any base language model or fine-tuned model without safety filtering, these models can easily be prompted by users to generate harmful and sensitive content. 
+- Such content may also be produced unintentionally, especially in cases involving bias, so we recommend that users consider the risks when applying this technology. 
+- Additionally, many statements from Ming Model or any LLM are often inaccurate, so facts should be verified.
+
+## License and use
+- Ming1.0 is built with Qwen-2.5-72B. Qwen-2.5-72B is licensed under the Qwen LICENSE AGREEMENT, Copyright (c) Alibaba Cloud. All Rights Reserved.
+- Subject to the Qwen LICENSE AGREEMENT, Ming1.0 is under MIT license.
+