metadata
language:
- en
tags:
- conversational
- interview
- dialogue-generation
license: apache-2.0
datasets:
- HuggingFaceFW/fineweb-edu
- agentlans/finewebedu-conversation
base_model: Qwen/Qwen2.5-1.5B-Instruct
pipeline_tag: text-generation
Qwen2.5-1.5B-Instruct-Conversation-Maker
Overview
A specialized variant of Qwen2.5-1.5B-Instruct fine-tuned for generating interview-style dialogues between a person and an expert. The model produces structured conversations in XML format for educational and AI applications.
Key Features:
- Structured Output: Generates
<conversation>blocks with<person>and<expert>roles. - Training Data: 9996 conversations derived from FineWebEdu, chunked at ~1950 Llama 3 tokens.
- Methodology: Used agentlans/Llama3.1-LexiHermes-SuperStorm and cognitivecomputations/Dolphin3.0-Llama3.2-3B for synthetic dialogue generation.
Usage
Input Format
Conversation:
{{YOUR_TEXT_HERE}}
Output Example
<conversation>
<person>What causes climate change?</person>
<expert>Human activities like burning fossil fuels release greenhouse gases...</expert>
...
</conversation>
Training Details
- Framework: LLaMA Factory
- Parameters: LoRA rank 16, alpha 32, rSLoRA, NEFTune (δ=5), dropout 0.2
- Epochs: 3
Limitations
- Context Gaps: May refer to entities outside the conversation like a table or a figure.
- Repetition: Occasional dull or redundant responses.
- Role Reversals: Expert/person labels may flip or be in the wrong order.
- Varying Quality: Depends on the length and formatting of the input data.