license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
LiteCoST: Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs
LiteCoST is a two-pillar framework designed to achieve both high accuracy and low latency for long-document question answering (QA) using Small Language Models (SLMs).
The model utilizes Chain-of-Structured-Thought (CoST), a schema-aware instruction template that guides models to produce both a step-wise reasoning trace and a structured output. This approach is distilled into compact models (3B/7B) through a two-stage fine-tuning process: Supervised Fine-Tuning (SFT) for structural alignment, followed by Group Relative Policy Optimization (GRPO) to enhance answer quality and process consistency.
Resources
- Paper: Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs
- GitHub Repository: HKUSTDial/LiteCoST
Method Overview
- Pillar 1: Chain-of-Structured-Thought (CoST): Guides a high-capability LLM to generate auditable traces that include structure analysis, trace generation, data verification, and refinement.
- Pillar 2: SLM Fine-Tuning: Compact models are trained on the CoST data using SFT and then optimized via GRPO with rewards for answer quality, formatting, and process consistency.
Performance
By distilling structure-first behavior into SLMs, this approach achieves LLM-comparable quality on multi-domain long-document QA while delivering significantly lower latency than GPT-4o and DeepSeek-R1 (671B).
Citation
@inproceedings{cost2026litecost,
title={Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs},
author={Liang, Seton and others},
booktitle={The Fourteenth International Conference on Learning Representations (ICLR)},
year={2026}
}