library_name: transformers
tags:
- qualitative-coding
- cpu
- research-artifact
extra_gated_prompt: >-
These checkpoints were trained on a qualitative-coding corpus whose
redistribution licence and participant-consent basis are not documented in the
retained research artifact (see "Data provenance" below). Access is granted
for non-commercial research use and reproduction of the accompanying paper
only. You are responsible for confirming that your own use is lawful in your
jurisdiction. Do not redistribute the weights or attempt to reconstruct the
underlying source corpus.
extra_gated_fields:
Name: text
Affiliation: text
Intended use: text
I will use these weights for non-commercial research only: checkbox
I will not redistribute the weights or reconstruct the source corpus: checkbox
extra_gated_button_content: Request access
Matched-Audit CPU Checkpoints
This repository contains every unique epoch-8 checkpoint from the paper's five-seed matched training and domain-composition experiments.
Layout
matched/tiny_hr/seed_*: human benchmark codes retained; technical reviews use machine codes.matched/tiny_mr/seed_*: benchmark codes replaced by machine codes; the same technical reviews are used.domain/single_hr/seed_*: benchmark-only human-code condition at matched update count.domain/single_mr/seed_*: benchmark-only machine-code condition at matched update count.
Seeds are 13, 42, 71, 101, and 137. Every folder is directly loadable with the
Transformers subfolder= argument. The mixed_hr and mixed_mr checkpoint
copies from the domain run are not uploaded twice: their hashes exactly match
the corresponding matched/tiny_hr and matched/tiny_mr files.
The checkpoints are research artifacts, not 20 independently recommended deployments. The human study used the seed-42 epoch-8 matched checkpoints. Automatic reference agreement should not be interpreted as qualitative validity.
Source-data permissions are still under review. Access through this gated repository does not authorize onward redistribution of its weights or training examples.
Warning: the tiny_mr checkpoints degenerate
The machine-code (tiny_mr) checkpoints repeat one of their own content words
in roughly 35% of benchmark outputs and 23% of app-review outputs, on every
seed ("Clarity and Clarity", "Motion for amputation of motion"), while scoring
close to tiny_hr on reference agreement. They are published as evidence for
that finding, not as usable coders.
Data provenance and release status
These weights derive from a 999-pair English open-coding benchmark: 600 pairs from social-science work across three university faculties (interviews and reviews, consensus-coded by three to five coders) and 399 SemEval-2014 Task 4 review excerpts, plus 1,990 machine-coded ICLR peer-review excerpts.
The supplied artifact records only the passage and its label. It does not record the original language, coder identities, adjudication trace, consent basis, or redistribution licence. Public availability of source text does not by itself establish permission to redistribute a compiled corpus or weights trained on it. Access is therefore gated, and the source passages are not released.
Memorisation
These are small sequence-to-sequence models fitted to a small label set, and they reproduce training annotations verbatim at a substantial rate. At the matched epoch-8 checkpoint, 28.9% of human-code (HR) outputs across 1,000 evaluated items are exact strings from the human training labels, drawing on 28 distinct labels. Treat generated codes as potentially reproducing the original coders' annotations rather than as novel interpretations. Long source passages are not recoverable from a model of this size; the annotation set is partially recoverable, which is why access is gated.
Intended use and limits
First-pass, editable code suggestions for one pre-segmented English passage, returning one code of at most six words. These models do not segment transcripts, assign multiple codes, build codebooks, or construct themes, and are not a substitute for a researcher. In a blinded five-expert evaluation, 47.5% of Tiny-HR suggestions were rated usable or better, against 85.0% for a task-adapted Qwen2.5-7B.
Citation
Accompanying paper: AI-Assisted Qualitative Coding on a CPU (under review). Code, prompts, analysis scripts and hashes accompany the submission.