---
license: mit
language:
- en
tags:
- text-classification
- app-reviews
- software-engineering
- roberta
pipeline_tag: text-classification
---

# IssueSpec V5 — App-Review Classifier (RoBERTa, 7-class)

The production Stage-1 classifier from the CIKM 2026 paper *IssueSpec: A
Framework for Structured Review-to-Issue Translation*. Fine-tuned RoBERTa head
that labels app-store reviews into the seven-class Maalej-Nabil taxonomy.

## Classes

`bug_report`, `feature_request`, `performance`, `usability`, `compatibility`,
`praise`, `other`

## Performance

On the 490-review expert gold standard:
- Cohen's κ = **0.592** (moderate; up from the V2 LLM baseline of 0.163)
- Accuracy = 65.0%, macro F1 = 0.653
- Recovers minority classes the LLM was blind to: compatibility F1 0.83, performance F1 0.77

V5 is trained on V2-corrected labels plus verified-anchor correction (5,230
expert-labeled reviews) and targeted compatibility augmentation (200 synthetic
+ 100 mined). See the paper §3.1 and §5.1 for full details.

## Usage

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tok = AutoTokenizer.from_pretrained("<ANON>/issuespec-v5-classifier")
model = AutoModelForSequenceClassification.from_pretrained("<ANON>/issuespec-v5-classifier")

text = "App crashes when opening ads on Samsung Galaxy S21 (Android 13)."
inputs = tok(text, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad():
    logits = model(**inputs).logits
pred = logits.argmax(-1).item()
print(model.config.id2label[pred])
```

## Cross-protocol generalization

Evaluated zero-shot against Maalej's 5,008 labels (out-of-distribution taxonomy):
macro F1 = 0.676, weighted F1 = 0.730, accuracy = 72.0% — confirming the
classifier performs concept recognition, not template memorization.

## Citation

Please cite the CIKM 2026 paper. Code, data, and the full V1–V5 checkpoint
series are linked from the project repository's `SETUP_GUIDE.md`.

## License

MIT