chrissoria/catllm-json-formatter

CatLLM JSON Formatter

A fine-tuned Qwen2.5-0.5B-Instruct model that converts messy LLM classification output into valid cat-llm JSON format.

Task

Given a list of numbered categories and raw (possibly malformed) classification output from another LLM, this model produces clean JSON:

{"1": "0", "2": "1", "3": "0", ...}

Usage

This model is used automatically by cat-llm when json_formatter=True:

import catllm as cat

results = cat.classify(
    input_data=df["responses"],
    categories=["Positive", "Negative", "Neutral"],
    api_key="your-key",
    json_formatter=True,  # enables the formatter fallback
)

Install the formatter dependencies: pip install cat-llm[formatter]

Training

• Base model: Qwen/Qwen2.5-0.5B-Instruct
• Method: LoRA (r=16, alpha=32) merged into base weights
• Training data: 8,000 synthetic examples covering 26+ messy output formats, with the category count spanning N=2..50 so the formatter reliably emits large (28- and 48-key) JSON objects, not just small ones.
• Epochs: 2
• Metrics: evaluated separately on low-N (<=12 categories) and high-N (>=25 categories) buckets; see the repository's eval gate for current numbers.

Prompt Format

The model uses the Qwen chat template with:

System: JSON formatter instructions (built into cat-llm)

User:

Categories:
1. Category A
2. Category B
...

Raw classification output:
{messy output here}

Assistant: {"1":"0","2":"1",...}