metadata
datasets:
- d3LLM/trajectory_data_dream_32
tags:
- diffusion
- text-generation
- fast-inference
- d3llm
pipeline_tag: text-generation
d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation 🚀
Model Description
d3LLM-Dream is an ultra-fast diffusion language model that achieves high generation speed while maintaining competitive performance. Built on the Dream architecture.
Key Features
- 🚀 High throughput: 4.5× faster than autoregressive models (Qwen-2.5-7B) on H100 GPU, 2.5× faster on A100 GPU. Achieves 235.34 tokens/s on H100 (vs 57.32 for AR baseline) on GSM8K-CoT Dataset.
- 📊 High AUP (Accuracy Under Parallelism) scores across benchmarks
- 🔧 Optimized for coding and math reasoning tasks
Usage
For detailed usage instructions, evaluation scripts, training datasets, and training code, please refer to the official GitHub repository and our blog:
- 👉 Code repo: https://github.com/hao-ai-lab/d3LLM
- 🌐 Blog: https://hao-ai-lab.github.io/blogs/text-diffusion/