Update README.md

README.md

@@ -3,4 +3,100 @@ license: mit
 base_model:
 - rubenroy/Zurich-7B-GCv2-5m
 library_name: transformers
----
+---
+# Maverick Model Card
+
+## Model Overview
+
+**Maverick** is a 14.7-billion-parameter causal language model fine-tuned from [Ruben Roy's Zurich-14B-GCv2-5m](https://huggingface.co/rubenroy/Zurich-14B-GCv2-5m). The base model, Zurich-14B-GCv2-5m, is itself a fine-tuned version of Alibaba's Qwen 2.5 14B Instruct model, trained on the GammaCorpus v2-5m dataset. Maverick is designed to excel in various STEM fields and general natural language processing tasks, offering enhanced reasoning and instruction-following capabilities.
+
+## Model Details
+
+- **Model Developer:** Aayan Mishra
+- **Model Type:** Causal Language Model
+- **Architecture:** Transformer with Rotary Position Embeddings (RoPE), SwiGLU activation, RMSNorm, and Attention QKV bias
+- **Parameters:** 14.7 billion total (13.1 billion non-embedding)
+- **Layers:** 48
+- **Attention Heads:** 28 for query and 4 for key-value (Grouped Query Attention)​
+- **Vocabulary Size:** Approximately 151,646 tokens
+- **Context Length:** Supports up to 131,072 tokens
+- **Languages Supported:** Over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, and Arabic
+- **License:** MIT
+
+## Training Details
+
+Maverick was fine-tuned using the Unsloth framework on a single NVIDIA A100 GPU. The fine-tuning process spanned approximately 90 minutes over 60 epochs, utilising a curated dataset focused on instruction-following and STEM-related content. This approach aimed to enhance the model's performance in complex reasoning and academic tasks.
+
+## Intended Use
+
+Maverick is designed for a range of applications, including but not limited to:
+
+- **STEM Reasoning:** Assisting with problem-solving and explanations in science, technology, engineering, and mathematics.
+- **Academic Assistance:** Providing support for tutoring, essay composition, and research inquiries.
+- **General NLP Tasks:** Engaging in text completion, summarisation, and question-answering tasks.
+- **Data Analysis:** Offering insights and interpretations of data-centric queries.
+
+While Maverick is a powerful tool for various applications, it is not intended for real-time, safety-critical systems or for processing sensitive personal information.
+
+## How to Use
+
+To utilize Maverick, ensure that you have the latest version of the `transformers` library installed:
+
+```bash
+pip install transformers
+```
+
+
+Here's an example of how to load the Maverick model and generate a response:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "Spestly/Maverick-1-7B"
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+prompt = "Explain the concept of entropy in thermodynamics."
+messages = [
+    {"role": "system", "content": "You are Maverick, an AI assistant designed to be helpful."},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=512
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+print(response)
+```
+
+## Limitations
+
+Users should be aware of the following limitations:
+
+- **Biases:** Maverick may exhibit biases present in its training data. Users should critically assess outputs, especially in sensitive contexts.
+- **Knowledge Cutoff:** The model's knowledge is current up to August 2024. It may not be aware of events or developments occurring after this date.
+- **Language Support:** While primarily trained on English data, performance in other languages may be inconsistent.
+
+## Acknowledgements
+
+Maverick builds upon the work of [Ruben Roy](https://huggingface.co/rubenroy), particularly the Zurich-14B-GCv2-5m model, which is a fine-tuned version of Alibaba's Qwen 2.5 14B Instruct model. Gratitude is also extended to the open-source AI community for their contributions to tools and frameworks that facilitated the development of Maverick.
+
+## License
+
+Maverick is released under the [MIT License](https://opensource.org/license/mit), permitting wide usage with proper attribution.