| --- |
| license: apache-2.0 |
| model-index: |
| - name: orca_mini_v7_7b |
| results: |
| - task: |
| type: text-generation |
| name: Text Generation |
| dataset: |
| name: IFEval (0-Shot) |
| type: HuggingFaceH4/ifeval |
| args: |
| num_few_shot: 0 |
| metrics: |
| - type: inst_level_strict_acc and prompt_level_strict_acc |
| value: 43.88 |
| name: strict accuracy |
| source: |
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=pankajmathur/orca_mini_v7_7b |
| name: Open LLM Leaderboard |
| - task: |
| type: text-generation |
| name: Text Generation |
| dataset: |
| name: BBH (3-Shot) |
| type: BBH |
| args: |
| num_few_shot: 3 |
| metrics: |
| - type: acc_norm |
| value: 33.95 |
| name: normalized accuracy |
| source: |
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=pankajmathur/orca_mini_v7_7b |
| name: Open LLM Leaderboard |
| - task: |
| type: text-generation |
| name: Text Generation |
| dataset: |
| name: MATH Lvl 5 (4-Shot) |
| type: hendrycks/competition_math |
| args: |
| num_few_shot: 4 |
| metrics: |
| - type: exact_match |
| value: 2.64 |
| name: exact match |
| source: |
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=pankajmathur/orca_mini_v7_7b |
| name: Open LLM Leaderboard |
| - task: |
| type: text-generation |
| name: Text Generation |
| dataset: |
| name: GPQA (0-shot) |
| type: Idavidrein/gpqa |
| args: |
| num_few_shot: 0 |
| metrics: |
| - type: acc_norm |
| value: 6.15 |
| name: acc_norm |
| source: |
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=pankajmathur/orca_mini_v7_7b |
| name: Open LLM Leaderboard |
| - task: |
| type: text-generation |
| name: Text Generation |
| dataset: |
| name: MuSR (0-shot) |
| type: TAUR-Lab/MuSR |
| args: |
| num_few_shot: 0 |
| metrics: |
| - type: acc_norm |
| value: 12.66 |
| name: acc_norm |
| source: |
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=pankajmathur/orca_mini_v7_7b |
| name: Open LLM Leaderboard |
| - task: |
| type: text-generation |
| name: Text Generation |
| dataset: |
| name: MMLU-PRO (5-shot) |
| type: TIGER-Lab/MMLU-Pro |
| config: main |
| split: test |
| args: |
| num_few_shot: 5 |
| metrics: |
| - type: acc |
| value: 35.19 |
| name: accuracy |
| source: |
| url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=pankajmathur/orca_mini_v7_7b |
| name: Open LLM Leaderboard |
| --- |
| |
| **Model Name: Qwen2 orca_mini_v7_7b** |
| # Qwen2 orca_mini_v7_7b is trained with various SFT Datasets |
| |
| <img src="https://huggingface.co/pankajmathur/orca_mini_v5_8b/resolve/main/orca_minis_small.jpeg" width="auto" /> |
| |
| <strong> |
| "Obsessed with GenAI's potential? So am I ! Let's create together 🚀 <a href="https://www.linkedin.com/in/pankajam" target="_blank">https://www.linkedin.com/in/pankajam</a>" |
| </strong> |
| |
| <br> |
| |
| ### NOTICE |
| By providing proper credit and attribution, you are granted permission to use this model as a foundational base for further Full fine tuning, DPO, PPO or ORPO tuning and any kind of Merges. |
| I actively encourage users to customize and enhance the model according to their specific needs, as this version is designed to be a comprehensive general model. |
| Dive in and innovate! |
| |
| ### Evaluation |
| Coming Soon.. |
| |
| |
| ### Example Usage |
| |
| Here is the ChatML prompt format |
| ``` |
| <|im_start|>system |
| You are Orca Mini, a helpful AI assistant.<|im_end|> |
| <|im_start|>user |
| Hello Orca Mini, what can you do for me?<|im_end|> |
| <|im_start|>assistant |
| ``` |
| Below shows a code example on how to use this model |
| |
| ```python |
| from transformers import AutoModel, AutoTokenizer |
| model_slug = "pankajmathur/orca_mini_v7_7b" |
| model = AutoModel.from_pretrained(model_slug) |
| tokenizer = AutoTokenizer.from_pretrained(model_slug) |
| messages = [ |
| {"role": "system", "content": "You are Orca Mini, a helpful AI assistant."}, |
| {"role": "user", "content": "Hello Orca Mini, what can you do for me?"} |
| ] |
| gen_input = tokenizer.apply_chat_template(messages, return_tensors="pt") |
| model.generate(**gen_input) |
| ``` |
| |
| **Quants** |
|
|
| GGUF : Coming Soon |
|
|
| AWQ: Coming Soon |
|
|
|
|
| ### Processing Long Texts (Based upon Qwen2-7B-Instruct suggestions at https://huggingface.co/Qwen/Qwen2-7B-Instruct) |
|
|
| To handle extensive inputs exceeding 32,768 tokens, we utilize [YARN](https://arxiv.org/abs/2309.00071), a technique for enhancing model length extrapolation, ensuring optimal performance on lengthy texts. |
|
|
| For deployment, we recommend using vLLM. You can enable the long-context capabilities by following these steps: |
|
|
| 1. **Install vLLM**: You can install vLLM by running the following command. |
|
|
| ```bash |
| pip install "vllm>=0.4.3" |
| ``` |
|
|
| Or you can install vLLM from [source](https://github.com/vllm-project/vllm/). |
|
|
| 2. **Configure Model Settings**: After downloading the model weights, modify the `config.json` file by including the below snippet: |
| ```json |
| { |
| "architectures": [ |
| "Qwen2ForCausalLM" |
| ], |
| // ... |
| "vocab_size": 152064, |
| // adding the following snippets |
| "rope_scaling": { |
| "factor": 4.0, |
| "original_max_position_embeddings": 32768, |
| "type": "yarn" |
| } |
| } |
| ``` |
| This snippet enable YARN to support longer contexts. |
| |
| 3. **Model Deployment**: Utilize vLLM to deploy your model. For instance, you can set up an openAI-like server using the command: |
|
|
| ```bash |
| python -u -m vllm.entrypoints.openai.api_server --model pankajmathur/orca_mini_v7_7b |
| ``` |
| Then you can access the Chat API by: |
| ```bash |
| curl http://localhost:8000/v1/chat/completions \ |
| -H "Content-Type: application/json" \ |
| -d '{ |
| "model": "pankajmathur/orca_mini_v7_7b", |
| "messages": [ |
| {"role": "system", "content": "You are Orca Mini, a helpful AI assistant."}, |
| {"role": "user", "content": "Hello Orca Mini, what can you do for me?"} |
| ] |
| }' |
| ``` |
| **Note**: Presently, vLLM only supports static YARN, which means the scaling factor remains constant regardless of input length, **potentially impacting performance on shorter texts**. We advise adding the `rope_scaling` configuration only when processing long contexts is required. |
| |
| # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) |
| Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_pankajmathur__orca_mini_v7_7b) |
|
|
| | Metric |Value| |
| |-------------------|----:| |
| |Avg. |22.41| |
| |IFEval (0-Shot) |43.88| |
| |BBH (3-Shot) |33.95| |
| |MATH Lvl 5 (4-Shot)| 2.64| |
| |GPQA (0-shot) | 6.15| |
| |MuSR (0-shot) |12.66| |
| |MMLU-PRO (5-shot) |35.19| |
|
|
| [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl) |
|
|
|
|
