| | --- |
| | license: apache-2.0 |
| | language: |
| | - en |
| | base_model: |
| | - Qwen/Qwen2.5-7B-Instruct |
| | pipeline_tag: text-generation |
| | library_name: transformers |
| | tags: |
| | - LCoT |
| | - Qwen |
| | - v2 |
| | datasets: |
| | - PowerInfer/QWQ-LONGCOT-500K |
| | - AI-MO/NuminaMath-CoT |
| | - prithivMLmods/Math-Solve |
| | - amphora/QwQ-LongCoT-130K |
| | - prithivMLmods/Deepthink-Reasoning |
| | model-index: |
| | - name: QwQ-LCoT2-7B-Instruct |
| | results: |
| | - task: |
| | type: text-generation |
| | name: Text Generation |
| | dataset: |
| | name: IFEval (0-Shot) |
| | type: wis-k/instruction-following-eval |
| | split: train |
| | args: |
| | num_few_shot: 0 |
| | metrics: |
| | - type: inst_level_strict_acc and prompt_level_strict_acc |
| | value: 55.76 |
| | name: averaged accuracy |
| | source: |
| | url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-LCoT2-7B-Instruct |
| | name: Open LLM Leaderboard |
| | - task: |
| | type: text-generation |
| | name: Text Generation |
| | dataset: |
| | name: BBH (3-Shot) |
| | type: SaylorTwift/bbh |
| | split: test |
| | args: |
| | num_few_shot: 3 |
| | metrics: |
| | - type: acc_norm |
| | value: 34.37 |
| | name: normalized accuracy |
| | source: |
| | url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-LCoT2-7B-Instruct |
| | name: Open LLM Leaderboard |
| | - task: |
| | type: text-generation |
| | name: Text Generation |
| | dataset: |
| | name: MATH Lvl 5 (4-Shot) |
| | type: lighteval/MATH-Hard |
| | split: test |
| | args: |
| | num_few_shot: 4 |
| | metrics: |
| | - type: exact_match |
| | value: 22.21 |
| | name: exact match |
| | source: |
| | url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-LCoT2-7B-Instruct |
| | name: Open LLM Leaderboard |
| | - task: |
| | type: text-generation |
| | name: Text Generation |
| | dataset: |
| | name: GPQA (0-shot) |
| | type: Idavidrein/gpqa |
| | split: train |
| | args: |
| | num_few_shot: 0 |
| | metrics: |
| | - type: acc_norm |
| | value: 6.38 |
| | name: acc_norm |
| | source: |
| | url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-LCoT2-7B-Instruct |
| | name: Open LLM Leaderboard |
| | - task: |
| | type: text-generation |
| | name: Text Generation |
| | dataset: |
| | name: MuSR (0-shot) |
| | type: TAUR-Lab/MuSR |
| | args: |
| | num_few_shot: 0 |
| | metrics: |
| | - type: acc_norm |
| | value: 15.75 |
| | name: acc_norm |
| | source: |
| | url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-LCoT2-7B-Instruct |
| | name: Open LLM Leaderboard |
| | - task: |
| | type: text-generation |
| | name: Text Generation |
| | dataset: |
| | name: MMLU-PRO (5-shot) |
| | type: TIGER-Lab/MMLU-Pro |
| | config: main |
| | split: test |
| | args: |
| | num_few_shot: 5 |
| | metrics: |
| | - type: acc |
| | value: 37.13 |
| | name: accuracy |
| | source: |
| | url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-LCoT2-7B-Instruct |
| | name: Open LLM Leaderboard |
| | --- |
| | |
| |
|
| |
|
| | # **QwQ-LCoT2-7B-Instruct** |
| |
|
| | The *QwQ-LCoT2-7B-Instruct* is a fine-tuned language model designed for advanced reasoning and instruction-following tasks. It leverages the Qwen2.5-7B base model and has been fine-tuned on the chain of thought reasoning datasets, focusing on chain-of-thought (CoT) reasoning for problems. This model is optimized for tasks requiring logical reasoning, detailed explanations, and multi-step problem-solving, making it ideal for applications such as instruction-following, text generation, and complex reasoning tasks. |
| |
|
| | # **Quickstart with Transformers** |
| |
|
| | Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents. |
| |
|
| | ```python |
| | from transformers import AutoModelForCausalLM, AutoTokenizer |
| | |
| | model_name = "prithivMLmods/QwQ-LCoT2-7B-Instruct" |
| | |
| | model = AutoModelForCausalLM.from_pretrained( |
| | model_name, |
| | torch_dtype="auto", |
| | device_map="auto" |
| | ) |
| | tokenizer = AutoTokenizer.from_pretrained(model_name) |
| | |
| | prompt = "How many r in strawberry." |
| | messages = [ |
| | {"role": "system", "content": "You are a helpful and harmless assistant. You are Qwen developed by Alibaba. You should think step-by-step."}, |
| | {"role": "user", "content": prompt} |
| | ] |
| | text = tokenizer.apply_chat_template( |
| | messages, |
| | tokenize=False, |
| | add_generation_prompt=True |
| | ) |
| | model_inputs = tokenizer([text], return_tensors="pt").to(model.device) |
| | |
| | generated_ids = model.generate( |
| | **model_inputs, |
| | max_new_tokens=512 |
| | ) |
| | generated_ids = [ |
| | output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) |
| | ] |
| | |
| | response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] |
| | ``` |
| |
|
| | # **Intended Use** |
| |
|
| | The QwQ-LCoT2-7B-Instruct model is designed for advanced reasoning and instruction-following tasks, with specific applications including: |
| |
|
| | 1. **Instruction Following**: Providing detailed and step-by-step guidance for a wide range of user queries. |
| | 2. **Logical Reasoning**: Solving problems requiring multi-step thought processes, such as math problems or complex logic-based scenarios. |
| | 3. **Text Generation**: Crafting coherent, contextually relevant, and well-structured text in response to prompts. |
| | 4. **Problem-Solving**: Analyzing and addressing tasks that require chain-of-thought (CoT) reasoning, making it ideal for education, tutoring, and technical support. |
| | 5. **Knowledge Enhancement**: Leveraging reasoning datasets to offer deeper insights and explanations for a wide variety of topics. |
| |
|
| | # **Limitations** |
| |
|
| | 1. **Data Bias**: As the model is fine-tuned on specific datasets, its outputs may reflect inherent biases from the training data. |
| | 2. **Context Limitation**: Performance may degrade for tasks requiring knowledge or reasoning that significantly exceeds the model's pretraining or fine-tuning context. |
| | 3. **Complexity Ceiling**: While optimized for multi-step reasoning, exceedingly complex or abstract problems may result in incomplete or incorrect outputs. |
| | 4. **Dependency on Prompt Quality**: The quality and specificity of the user prompt heavily influence the model's responses. |
| | 5. **Non-Factual Outputs**: Despite being fine-tuned for reasoning, the model can still generate hallucinated or factually inaccurate content, particularly for niche or unverified topics. |
| | 6. **Computational Requirements**: Running the model effectively requires significant computational resources, particularly when generating long sequences or handling high-concurrency workloads. |
| | # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) |
| | Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/prithivMLmods__QwQ-LCoT2-7B-Instruct-details)! |
| | Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=prithivMLmods%2FQwQ-LCoT2-7B-Instruct&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)! |
| |
|
| | | Metric |Value (%)| |
| | |-------------------|--------:| |
| | |**Average** | 28.60| |
| | |IFEval (0-Shot) | 55.76| |
| | |BBH (3-Shot) | 34.37| |
| | |MATH Lvl 5 (4-Shot)| 22.21| |
| | |GPQA (0-shot) | 6.38| |
| | |MuSR (0-shot) | 15.75| |
| | |MMLU-PRO (5-shot) | 37.13| |
| |
|
| |
|
