| --- |
| license: apache-2.0 |
| library_name: transformers |
| pipeline_tag: text-generation |
| --- |
| |
| # LiteCoST: Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs |
|
|
| LiteCoST is a two-pillar framework designed to achieve both high accuracy and low latency for long-document question answering (QA) using Small Language Models (SLMs). |
|
|
| The model utilizes **Chain-of-Structured-Thought (CoST)**, a schema-aware instruction template that guides models to produce both a step-wise reasoning trace and a structured output. This approach is distilled into compact models (3B/7B) through a two-stage fine-tuning process: Supervised Fine-Tuning (SFT) for structural alignment, followed by Group Relative Policy Optimization (GRPO) to enhance answer quality and process consistency. |
|
|
| ## Resources |
| - **Paper:** [Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs](https://huggingface.co/papers/2603.29232) |
| - **GitHub Repository:** [HKUSTDial/LiteCoST](https://github.com/HKUSTDial/LiteCoST) |
|
|
| ## Method Overview |
|
|
| 1. **Pillar 1: Chain-of-Structured-Thought (CoST):** Guides a high-capability LLM to generate auditable traces that include structure analysis, trace generation, data verification, and refinement. |
| 2. **Pillar 2: SLM Fine-Tuning:** Compact models are trained on the CoST data using SFT and then optimized via GRPO with rewards for answer quality, formatting, and process consistency. |
|
|
| ## Performance |
| By distilling structure-first behavior into SLMs, this approach achieves LLM-comparable quality on multi-domain long-document QA while delivering significantly lower latency than GPT-4o and DeepSeek-R1 (671B). |
|
|
| ## Citation |
| ```bibtex |
| @inproceedings{cost2026litecost, |
| title={Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs}, |
| author={Liang, Seton and others}, |
| booktitle={The Fourteenth International Conference on Learning Representations (ICLR)}, |
| year={2026} |
| } |
| ``` |
