|
|
--- |
|
|
library_name: transformers |
|
|
base_model: |
|
|
- Qwen/Qwen2.5-3B-Instruct |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- amphora/QwQ-LongCoT-130K |
|
|
- amphora/QwQ-LongCoT-130K-2 |
|
|
- amphora/verfiable-25k |
|
|
- amphora/m-math500 |
|
|
language: |
|
|
- en |
|
|
- zh |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- Math |
|
|
- Code |
|
|
- Thinker |
|
|
- Reasoning |
|
|
- 3B |
|
|
- QwQ |
|
|
- Mini |
|
|
- text-generation-inference |
|
|
- SFT |
|
|
- trl |
|
|
--- |
|
|
|
|
|
 |
|
|
|
|
|
# **PocketThinker-QwQ-3B-Instruct** |
|
|
|
|
|
> PocketThinker-QwQ-3B-Instruct is based on the Qwen2.5-3B-Instruct architecture, designed as a lightweight and efficient reasoning assistant. It serves as the pocket-sized version of QwQ-LCoT-7B-Instruct, optimized for fast inference while maintaining strong problem-solving and computational capabilities. This model is fine-tuned for enhanced structured reasoning, minimal token wastage, and high-quality technical responses. |
|
|
|
|
|
## **Key Improvements** |
|
|
1. **Optimized for Coding**: Specializes in generating structured, efficient code with minimal redundancy for smooth execution. |
|
|
2. **Compact yet Powerful**: Maintains strong problem-solving capabilities within a smaller 3B parameter architecture, ensuring accessibility on resource-limited devices. |
|
|
3. **Advanced Reasoning Capabilities**: Excels in algorithmic problem-solving, mathematical reasoning, and structured technical explanations. |
|
|
4. **Efficient Memory Utilization**: Reduces computational overhead while maintaining high-quality outputs. |
|
|
5. **Focused Output Generation**: Avoids unnecessary token generation, ensuring concise and relevant responses. |
|
|
|
|
|
## **Quickstart with transformers** |
|
|
|
|
|
Here is a code snippet to load the tokenizer and model using `apply_chat_template` for structured input formatting: |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
model_name = "prithivMLmods/PocketThinker-QwQ-3B-Instruct" |
|
|
|
|
|
model = AutoModelForCausalLM.from_pretrained( |
|
|
model_name, |
|
|
torch_dtype="auto", |
|
|
device_map="auto" |
|
|
) |
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
|
|
|
prompt = "Write a Python function to find the Fibonacci sequence." |
|
|
messages = [ |
|
|
{"role": "system", "content": "You are an advanced coding assistant."}, |
|
|
{"role": "user", "content": prompt} |
|
|
] |
|
|
text = tokenizer.apply_chat_template( |
|
|
messages, |
|
|
tokenize=False, |
|
|
add_generation_prompt=True |
|
|
) |
|
|
model_inputs = tokenizer([text], return_tensors="pt").to(model.device) |
|
|
|
|
|
generated_ids = model.generate( |
|
|
**model_inputs, |
|
|
max_new_tokens=6090 |
|
|
) |
|
|
generated_ids = [ |
|
|
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) |
|
|
] |
|
|
|
|
|
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] |
|
|
print(response) |
|
|
``` |
|
|
|
|
|
## **Intended Use** |
|
|
1. **Code Generation & Optimization**: |
|
|
Supports developers in writing, refining, and optimizing code across multiple programming languages. |
|
|
2. **Algorithm & Mathematical Problem Solving**: |
|
|
Delivers precise solutions and structured explanations for complex problems. |
|
|
3. **Technical Documentation & Explanation**: |
|
|
Assists in generating well-structured documentation for libraries, APIs, and coding concepts. |
|
|
4. **Debugging Assistance**: |
|
|
Helps identify and correct errors in code snippets. |
|
|
5. **Educational Support**: |
|
|
Simplifies programming topics for students and learners with clear explanations. |
|
|
6. **Structured Data Processing**: |
|
|
Generates structured outputs like JSON, XML, and tables for data science applications. |
|
|
|
|
|
## **Limitations** |
|
|
1. **Hardware Constraints**: |
|
|
Although lighter than larger models, still requires a moderately powerful GPU or TPU for optimal performance. |
|
|
2. **Potential Bias in Responses**: |
|
|
Outputs may reflect biases present in training data. |
|
|
3. **Limited Creativity**: |
|
|
May generate variable results in non-technical, creative tasks. |
|
|
4. **No Real-Time Awareness**: |
|
|
Lacks access to real-world events beyond its training cutoff. |
|
|
5. **Error Propagation in Long Responses**: |
|
|
Minor mistakes in early outputs may affect overall coherence in lengthy responses. |
|
|
6. **Prompt Sensitivity**: |
|
|
The effectiveness of responses depends on well-structured prompts. |
