|
|
--- |
|
|
language: |
|
|
- en |
|
|
tags: |
|
|
- deepseek |
|
|
- paraphrase |
|
|
- lora |
|
|
- text-generation |
|
|
license: mit |
|
|
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B |
|
|
datasets: |
|
|
- quora |
|
|
model-index: |
|
|
- name: Deepseek Paraphrase |
|
|
results: [] |
|
|
--- |
|
|
|
|
|
# Deepseek Paraphrase |
|
|
|
|
|
This model is a fine-tuned version of [deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) that has been specialized for high-quality paraphrase generation. It was trained using LoRA (Low-Rank Adaptation) and then merged back into the base model for efficient inference. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Base Model**: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B |
|
|
- **Task**: Paraphrase Generation |
|
|
- **Training Method**: LoRA fine-tuning with r=16, alpha=32 |
|
|
- **Training Data**: Multi-domain text from literary works, technical documentation, academic papers, and articles, plus the Quora paraphrase dataset |
|
|
|
|
|
## Performance |
|
|
|
|
|
This model outperforms standard paraphrasing models like BART and T5 on key metrics: |
|
|
- **Semantic Preservation** (BERTScore): 0.952 - Excellent |
|
|
- **Lexical Diversity** (BLEU Diversity): 0.513 - Acceptable |
|
|
- **Character-level Changes** (Edit Distance): 0.344 - Acceptable |
|
|
- **Structural Variation** (Syntactic Diversity): 0.147 - Moderate |
|
|
- **Overall Balance** (Harmonic Score): 0.468 - Acceptable |
|
|
|
|
|
## Usage |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
model_name = "PeterAM4/deepseek-paraphrase" |
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
model = AutoModelForCausalLM.from_pretrained(model_name) |
|
|
|
|
|
text = "Learn Once, Write Anywhere: We don't make assumptions about the rest of your technology stack, so you can develop new features in React without rewriting existing code." |
|
|
|
|
|
prompt = f"<｜begin▁of▁sentence｜><｜User｜>Paraphrase the following text while preserving its meaning but changing the wording and structure: {text}<｜Assistant｜><think>\nLet me analyze this text and find ways to rephrase it while keeping the same meaning.\nI need to use different vocabulary and structure.\n</think>\n\n" |
|
|
|
|
|
inputs = tokenizer(prompt, return_tensors="pt").to(model.device) |
|
|
outputs = model.generate( |
|
|
**inputs, |
|
|
max_new_tokens=200, |
|
|
temperature=0.7, |
|
|
top_p=0.95, |
|
|
do_sample=True |
|
|
) |
|
|
|
|
|
paraphrase = tokenizer.decode(outputs[0], skip_special_tokens=True).replace(prompt, "") |
|
|
print(paraphrase) |
|
|
``` |
|
|
|
|
|
## Limitations |
|
|
|
|
|
- Very technical or domain-specific terminology may not be paraphrased optimally |
|
|
- Always review paraphrases for factual accuracy and meaning preservation |
|
|
|
|
|
## Citation |
|
|
|
|
|
If you use this model in your research or applications, please cite: |
|
|
|
|
|
``` |
|
|
@misc{deepseek-paraphrase, |
|
|
author = {PeterAM4}, |
|
|
title = {DeepSeek Paraphrase: Fine-tuned DeepSeek model for high-quality paraphrasing}, |
|
|
year = {2025}, |
|
|
publisher = {Hugging Face}, |
|
|
howpublished = {\url{https://huggingface.co/PeterAM4/deepseek-paraphrase}} |
|
|
} |
|
|
``` |
|
|
|
