README.md

---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
---

# LiteCoST: Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs

LiteCoST is a two-pillar framework designed to achieve both high accuracy and low latency for long-document question answering (QA) using Small Language Models (SLMs). 

The model utilizes **Chain-of-Structured-Thought (CoST)**, a schema-aware instruction template that guides models to produce both a step-wise reasoning trace and a structured output. This approach is distilled into compact models (3B/7B) through a two-stage fine-tuning process: Supervised Fine-Tuning (SFT) for structural alignment, followed by Group Relative Policy Optimization (GRPO) to enhance answer quality and process consistency.

## Resources
- **Paper:** [Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs](https://huggingface.co/papers/2603.29232)
- **GitHub Repository:** [HKUSTDial/LiteCoST](https://github.com/HKUSTDial/LiteCoST)

## Method Overview

1. **Pillar 1: Chain-of-Structured-Thought (CoST):** Guides a high-capability LLM to generate auditable traces that include structure analysis, trace generation, data verification, and refinement.
2. **Pillar 2: SLM Fine-Tuning:** Compact models are trained on the CoST data using SFT and then optimized via GRPO with rewards for answer quality, formatting, and process consistency.

## Performance
By distilling structure-first behavior into SLMs, this approach achieves LLM-comparable quality on multi-domain long-document QA while delivering significantly lower latency than GPT-4o and DeepSeek-R1 (671B).

## Citation
```bibtex
@inproceedings{cost2026litecost,
  title={Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs},
  author={Liang, Seton and others},
  booktitle={The Fourteenth International Conference on Learning Representations (ICLR)},
  year={2026}
}
```