| --- |
| license: llama2 |
| language: |
| - en |
| tags: |
| - mistral |
| - merge |
| library_name: transformers |
| pipeline_tag: text-generation |
| mergekit: |
| - Weyaxi/OpenHermes-2.5-neural-chat-v3-3-openchat-3.5-1210-Slerp |
| - uukuguy/speechless-mistral-six-in-one-7b |
| datasets: |
| - stingning/ultrachat |
| - garage-bAInd/Open-Platypus |
| - Open-Orca/OpenOrca |
| - TIGER-Lab/MathInstruct |
| - OpenAssistant/oasst_top1_2023-08-25 |
| - teknium/openhermes |
| - meta-math/MetaMathQA |
| - Open-Orca/SlimOrca |
|
|
| --- |
| |
| <p align="center"> |
| <img src="https://codeberg.org/aninokuma/DeydooAssistant/raw/branch/main/logo.webp" height="256px" alt="SynthIQ"> |
| </p> |
|
|
| # SynthIQ |
|
|
| This is SynthIQ, rated 92.23/100 by GPT-4 across varied complex prompts. I used [mergekit](https://github.com/cg123/mergekit) to merge models. |
|
|
| Metrics from OpenLLM leaderboard: |
|
|
| | Model | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K | |
| | ---------------------------------------- | ------- | ----- | --------- | ----- | ---------- | ---------- | ------ | |
| | Weyaxi/OpenHermes-2.5_neural-chat-v3-3-openchat-5-1210-Slerp | 71.26 | 67.92 | 86.32 | 65.47 | 56.45 | 79.72 | 71.72 | |
| | sethuiyer/SynthIO-7b | 69.37 | 65.87 | 85.82 | 64.75 | 57 | 78.69 | 64.06 | |
| | uukuguy/speechless-mistral-six-in-one-7b | 60.76 | 62.97 | 84.6 | 63.29 | 57.77 | 77.51 | 18.42 | |
| |
| |
| # Yaml Config |
| |
| ```yaml |
| |
| slices: |
| - sources: |
| - model: Weyaxi/OpenHermes-2.5-neural-chat-v3-3-openchat-3.5-1210-Slerp |
| layer_range: [0, 32] |
| - model: uukuguy/speechless-mistral-six-in-one-7b |
| layer_range: [0, 32] |
| |
| merge_method: slerp |
| base_model: mistralai/Mistral-7B-v0.1 |
|
|
| parameters: |
| t: |
| - filter: self_attn |
| value: [0, 0.5, 0.3, 0.7, 1] |
| - filter: mlp |
| value: [1, 0.5, 0.7, 0.3, 0] |
| - value: 0.5 # fallback for rest of tensors |
| tokenizer_source: union |
|
|
| dtype: bfloat16 |
|
|
| ``` |
| |
| <!-- prompt-template start --> |
| ## Prompt template: ChatML |
| |
| ``` |
| <|im_start|>system |
| {system_message}<|im_end|> |
| <|im_start|>user |
| {prompt}<|im_end|> |
| <|im_start|>assistant |
|
|
| ``` |
| |
| <!-- prompt-template end --> |
| |
| SynthIQ's strengths can be succinctly summarized as follows: |
| |
| 1. **Advanced Natural Language Processing**: SynthIQ excels in understanding and generating natural language, making it highly effective for conversational AI applications. |
| |
| 2. **Strong Commonsense Reasoning**: It demonstrates a solid grasp of everyday scenarios and contexts, essential for practical and real-world applications. |
| |
| 3. **Creative and Engaging Content Generation**: SynthIQ has the capability to produce creative content, useful in fields like marketing, creative writing, and social media engagement. |
| |
| 4. **Adaptive User Interaction**: It can effectively adapt to various user personas, providing personalized experiences and recommendations. |
| |
| 5. **Multitasking Across Languages and Subjects**: SynthIQ is adept at handling tasks across different languages and subjects, showcasing its versatility in global and multifaceted settings. |
| |
| 6. **Analytical and Problem-Solving Skills**: The model shows proficiency in analytical reasoning and problem-solving, applicable in data-driven decision-making and complex scenario analysis. |
| |
| 7. **Cultural and Contextual Awareness**: SynthIQ's awareness of different cultural and social contexts makes it suitable for applications requiring cultural sensitivity. |
| |
| 8. **Empathetic and Human-Like Interactions**: The model can engage in empathetic and human-like dialogues, ideal for applications in mental health support, customer service, and education. |
| |
| |
| License is LLama2 license as uukuguy/speechless-mistral-six-in-one-7b is llama2 license. |
| # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) |
| Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_sethuiyer__SynthIQ-7b) |
| |
| | Metric | Value | |
| |-----------------------|---------------------------| |
| | Avg. | 69.37 | |
| | ARC (25-shot) | 65.87 | |
| | HellaSwag (10-shot) | 85.82 | |
| | MMLU (5-shot) | 64.75 | |
| | TruthfulQA (0-shot) | 57.0 | |
| | Winogrande (5-shot) | 78.69 | |
| | GSM8K (5-shot) | 64.06 | |
| |
