Update README.md

README.md

@@ -1,38 +1,25 @@
 ---
 language:
 - en
+pipeline_tag: text-generation
 tags:
-- physics
+- llama-3.1
 - astronomy
 - astrophysics
 - cosmology
-license:
-- llama3.1
+- arxiv
+inference: false
 base_model:
 - meta-llama/Meta-Llama-3.1-8B
-library_name: transformers
 ---
 
 # AstroSage-Llama-3.1-8B
 
-<INSERT PAPER LINK HERE>
-
-AstroSage-Llama-3.1-8B is a domain-specialized natural-language AI assistant
-tailored for research in astronomy, astrophysics, and cosmology. Trained on the
-complete collection of astronomy-related arXiv papers from 2007-2024 along with
-millions of synthetically-generated question-answer pairs and other
-astronomical literature, AstroSage-Llama-3.1-8B demonstrates excellent
-proficiency on a wide range of questions. AstroSage-Llama-3.1-8B scores 80.9%
-on the AstroMLab-1 benchmark, greatly outperforming all models---proprietary
-and open-weight---in the 8-billion parameter class, and performing on par with
-GPT-4o. This achievement demonstrates the potential of domain specialization in
-AI, suggesting that focused training can yield capabilities exceeding those of
-much larger, general-purpose models. AstroSage-Llama-3.1-8B is freely
-available, enabling widespread access to advanced AI capabilities for
-astronomical education and research.
+AstroSage-Llama-3.1-8B is a domain-specialized natural-language AI assistant tailored for research in astronomy, astrophysics, and cosmology. Trained on the complete collection of astronomy-related arXiv papers from 2007-2024 along with millions of synthetically-generated question-answer pairs and other astronomical literature, AstroSage-Llama-3.1-8B demonstrates excellent proficiency on a wide range of questions. This achievement demonstrates the potential of domain specialization in AI, suggesting that focused training can yield capabilities exceeding those of much larger, general-purpose models.
 
 ## Model Details
-- **Model Type**: Domain-specialized LLM
+
+- **Base Architecture**: Meta-Llama-3.1-8B
 - **Base Model**: Meta-Llama-3.1-8B
 - **Parameters**: 8 billion
 - **Training Focus**: Astronomy, Astrophysics, Cosmology, and Astronomical Instrumentation
@@ -42,29 +29,72 @@ astronomical education and research.
   2. Supervised Fine-tuning (SFT) on QA pairs and instruction sets
   3. Model merging with Meta-Llama-3.1-8B-Instruct (75% CPT+SFT / 25% Meta-Instruct)
 
-## Performance
-- **AstroMLab-1 Benchmark**: 80.9% accuracy
-  - Outperforms all 8B parameter models
-  - Comparable to GPT-4o (80.4%)
-  - ~1000x more cost-effective than proprietary models
-  - 8 percentage-point improvement over base Llama-3.1-8b model on Astronomy Q&A benchmark
+## Using the model
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Load the model and tokenizer
+model = AutoModelForCausalLM.from_pretrained("AstroMLab/AstroSage-8b", device_map="auto")
+tokenizer = AutoTokenizer.from_pretrained("AstroMLab/AstroSage-8b")
+
+# Function to generate a response
+def generate_response(prompt):
+    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+    outputs = model.generate(
+        **inputs,
+        max_new_tokens=128,
+        do_sample=True,
+        pad_token_id=tokenizer.eos_token_id,
+    )
+    response = outputs[0][inputs['input_ids'].shape[-1]:]
+    decoded = tokenizer.decode(response, skip_special_tokens=True)
+
+    return decoded
+
+# Example usage
+prompt = """
+You are an expert in general astrophysics. Your task is to answer the following question:
+What are the main components of a galaxy?
+"""
+response = generate_response(prompt)
+print(response)
+```
 
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/643f1ddce2ea47d170103537/9cXdGN7xf4vDag0_bgJcL.png)
 
-- **General Capabilities**: Maintains strong performance on standard benchmarks
-  - IF-EVAL: 41.4%
-  - BBH: 52.9%
-  - MATH: 8.4%
-  - GPQA: 31.2%
-  - MUSR: 38.9%
-  - MMLU-PRO: 34.6%
+## Model Improvements and Performance
+
+AstroSage-Llama-3.1-8B shows remarkable performance improvements:
+
+| Model | Score (%) |
+|-------|-----------|
+| **AstroSage-Llama-3.1-8B** | **80.9** |
+| GPT-4o | 80.4 |
+| LLaMA-3-8B | 72.9 |
+| Gemma-2-9B | 71.5 |
+| Qwen-2.5-7B | 70.4 |
+| Yi-1.5-9B | 68.4 |
+| InternLM-2.5-7B | 64.5 |
+| Mistral-7B-v0.3 | 63.9 |
+| ChatGLM3-6B | 50.4 |
+
+The model demonstrates:
+- Outperformance of all 8B parameter models
+- Comparable performance to GPT-4o (80.4%)
+- ~1000x more cost-effective than proprietary models
+- 8 percentage-point improvement over base Llama-3.1-8b model
+
 
 ## Training Data
+
 - **Continued Pre-training**:
   - ~250,000 arXiv preprints (2007-2024) from astro-ph and gr-qc
   - Astronomy-related Wikipedia articles
   - Selected astronomy textbooks
   - Total: 3.3 billion tokens, 19.9 GB plaintext
+
 - **Supervised Fine-tuning**:
   - 8.8 million curated QA pairs
   - Filtered Infinity-Instruct-7M dataset
@@ -87,16 +117,20 @@ astronomical education and research.
 - Performance primarily validated on multiple-choice questions
 - Primarily trained for use in English
 
-## Ethical Considerations
-- Should not be used as sole source for critical research decisions
-- Output should be verified against primary sources
-- May reflect biases present in astronomical literature
-
 ## Technical Specifications
 - Architecture: Based on Meta-Llama 3.1
 - Training Infrastructure: ORNL OLCF Frontier
 - Hosting: Hugging Face Hub (AstroMLab/AstroSage-8B)
 
+## Ethical Considerations
+
+While this model is designed for scientific use:
+- Should not be used as sole source for critical research decisions
+- Output should be verified against primary sources
+- May reflect biases present in astronomical literature
+
 ## Citation and Contact
- - Contract: Corresponding author Tijmen de Haan, email: tijmen dot dehaan at gmail dot com and AstroMLab astromachinelearninglab at gmail dot com
- - Please cite the AstroMLab 3 paper when referencing to this model.
+
+- Corresponding author: Tijmen de Haan (tijmen dot dehaan at gmail dot com)
+- AstroMLab: astromachinelearninglab at gmail dot com
+- Please cite the AstroMLab 3 paper when referencing this model