paper-classifier-2B-v2

Classify scientific papers into a user-supplied taxonomy, or judge whether a topic is central to a paper. Returns probabilities in one forward pass, without generating text. Made for Science Made Readable.

Use the supplied v2 helper. List questions MUST put the taxonomy before the paper. The earlier public model used a different, paper-first format. Its helper is not compatible with these weights. Yes/no questions intentionally keep the paper first and the topic afterward. Do not reorder that prompt either.

• Size tier: 2B, derived from Qwen3.5-2B. Sibling: paper-classifier-0.8B-v2.
• Standalone BF16 safetensors, not a LoRA adapter. Load with AutoModelForCausalLM, not AutoModelForSequenceClassification. This is not an NLI model; the generic Hugging Face zero-shot-classification pipeline is not the supported interface.
• Scope: scientific title/abstract classification, custom scientific labels, topic membership. Not a general chat model, embedding model, or reliable classifier for arbitrary non-research text.
• Exported weights contain 2,390,384,448 stored parameters (4.78 GB): the size tier follows the upstream name; the export includes an untied vocabulary-output matrix.
• License: Apache 2.0, including the inference helpers. Training data and the data-generation recipe are not published with this release. No claim of open training-data availability is made.

Quick start: Transformers

pip install "transformers==5.3.0" torch huggingface_hub safetensors

The snippets fetch a small Python helper explicitly; inspect it before importing, as with any downloaded code. The model itself does not require trust_remote_code=True. Pin revision="v2.0.0" for reproducibility.

from huggingface_hub import hf_hub_download
import importlib.util

repo = "maximerivest/paper-classifier-2B-v2"
revision = "v2.0.0"
path = hf_hub_download(repo, "paper_classifier.py", revision=revision)
spec = importlib.util.spec_from_file_location("paper_classifier", path)
pc = importlib.util.module_from_spec(spec)
spec.loader.exec_module(pc)

clf = pc.PaperClassifier(repo, revision=revision)
paper = pc.paper_text(
    "Habitat fragmentation and tropical forest birds",
    "We surveyed bird communities across 40 forest fragments. Species richness "
    "declined with fragment area and increased with connectivity.",
)
print(clf.classify(
    paper,
    ["Ecology", "Ornithology", "Particle physics", "Economics"],
    scheme="Fields of science",
))
# Measured with this model (rounding and hardware may change the last digits):
# {'Ornithology': 0.724677, 'Ecology': 0.275036, 'Particle physics': 0.000159, 'Economics': 0.000128}

print(clf.is_on_topic(paper, "habitat fragmentation"))
# approximately 0.999988

CPU loading works with float32 by default, but is slower. On CUDA the default is bfloat16. batch_size controls Transformers memory use; lower it if a batch does not fit. The Transformers helper does not reuse prefix states.

API: inputs and outputs

classify(paper: str, options: list, scheme: str, question: str, ...) -> dict[str, float]
classify_many(papers: list[str], options: list, scheme: str, question: str, ...) -> list[dict[str, float]]
is_on_topic(paper: str, topic: str, definition: str | None = None) -> float
is_on_topic_many(papers: list[str], topic: str, definition: str | None = None) -> list[float]

Options are strings, or dictionaries with name, optional about, optional parent, optional id:

options = [
    {"id": "eco", "name": "Ecology", "about": "Organisms, populations and their environment", "parent": "Biology"},
    {"id": "opt", "name": "Optics", "about": "Light, optical phenomena and optical methods", "parent": "Physics"},
]
p = clf.classify(paper, options, scheme="My scientific taxonomy")
# {"eco": ..., "opt": ...}; keys are IDs if supplied, otherwise option names.

p_yes = clf.is_on_topic(
    paper, "habitat fragmentation",
    definition="Effects of splitting habitat into smaller or isolated patches; not habitat loss alone.",
)

• Choice probabilities sum to one over the supplied options. They are relative best-fit probabilities, not independent membership probabilities or a guarantee that any option is appropriate.
• Yes/no returns P(yes | yes, no) for central topic membership. A paper may be central to several topics.
• Names/IDs must be unique. Supply distinct IDs if two options have the same name.
• none_option=True adds an explicit rejection option, returned under "__none__". Rejection remains unreliable on unfamiliar/non-scientific text; do not treat this switch as a dependable out-of-domain detector.
• orders=3 averages three option orders. It costs three passes and is optional: improvement is not guaranteed.
• describe=False removes option descriptions; parents=False removes parent names. These are explicit quality/speed trade-offs, not automatic reductions made silently by the helper.
• The codebook has 562 single-token codes. This is a token-interface limit, not a claim of reliable accuracy with 562 long descriptions. Lists must also fit the prompt budget; most measured long-list tests use 149–176 options. The helper raises on overflow. Shorten descriptions or split the taxonomy; do not truncate away options.
• For large hierarchies, classify one level at a time, or classify a feasible leaf list and sum its probabilities into parents. Direct broad-field and hierarchical routing accuracy can be weaker than flat classification.

Fast bulk use: taxonomy first, shared-prefix caching

Use this when many papers share the same taxonomy. The vLLM helper reuses the taxonomy's full-attention and recurrent states; the paper is the changing suffix. It reads final hidden states and projects onto the offered answer codes—no generated tokens and no 128-logprob limit.

Tested with vLLM 0.29.0. Install a matching supported CUDA/PyTorch build for your platform; keep it in a separate environment if its dependencies conflict with the Transformers-only environment.

pip install "vllm==0.29.0" huggingface_hub safetensors numpy

from huggingface_hub import snapshot_download
import sys

repo = "maximerivest/paper-classifier-2B-v2"
revision = "v2.0.0"
helpers = snapshot_download(repo, revision=revision,
                            allow_patterns=["paper_classifier.py", "vllm_fast.py"])
sys.path.insert(0, helpers)
from vllm_fast import FastPaperClassifier
from paper_classifier import paper_text

clf = FastPaperClassifier(repo, revision=revision, gpu_memory_utilization=0.8)
papers = [
    paper_text("Bird diversity in forest fragments", "We measured species richness and habitat connectivity."),
    paper_text("A liquid xenon dark matter search", "We report limits on weakly interacting massive particles."),
]
print(clf.classify_many(papers, ["Ecology", "Particle physics"], scheme="Fields of science"))

Prefix reuse benefits repeated taxonomies. It does not make arbitrary new lists free, and yes/no still uses the paper-first prompt. Do not transplant speed/cost numbers from the older public models to this release. Hardware, list length, batch size, cache hits, library version and numerical settings all matter.

Exact prompt and score contract

System message (shared by both tasks, verbatim):

You classify scientific papers. You get a paper (its title, and its abstract when available), then a question. For a list of options, answer with the code of the option that best describes the research content of the paper, following the classification's own meaning of each option. For a single topic, answer yes if the topic is a central subject of the paper, else no. On topic: the paper studies the topic itself as a main subject: the phenomenon, its causes, consequences, mechanisms, measurement, methods, or how it is managed, in any system or organism. Not on topic: the topic is absent or only mentioned in passing; or the words mean something else in this paper; or the paper studies a neighbouring subject without studying the topic itself.

Choice user message, conceptually:

## Classification: <scheme name and meaning>
<question>
Options:
AA. <option 1 name> — <optional about>; in <optional parent>
AB. <option 2 name> — <optional about>; in <optional parent>
...
<cache-alignment newline tokens>
## Paper
Title: <title>
Venue: <optional venue>
Year: <optional year>
Abstract: <abstract, or (no abstract)>

Answer with the option code only.

Yes/no user message, verbatim structure:

<paper text>

## Topic: <topic> (what it covers: <optional definition>)
Is this topic a central subject of the paper? Answer yes or no.

Both use the tokenizer's chat template with add_generation_prompt=True, enable_thinking=False. The helper constructs token IDs in pieces and inserts individual newline tokens so the shared choice prefix ends on a cache block boundary (544 tokens with the tested vLLM configuration). It never puts the list after the paper.

paper_text() caps the abstract at 1,500 characters to match the benchmark inputs. The inference helpers cap the paper portion at 700 tokens by default and the whole prompt at 4,000 tokens. These caps concern supplied text, not full-document understanding. The helpers preserve the entire offered list and answer prefix or raise an error.

The output is computed as:

# h: final hidden state after the assistant prefix; W: this model's lm_head.weight
logits = h.float() @ W[offered_answer_token_ids].float().T
probabilities = logits.softmax(dim=-1)

For choices use classifier.json's code_ids[:number_of_options]; for yes/no use [yes, no]. Do not use greedy generated text, all-vocabulary softmax, or token IDs from another tokenizer. The first code is not necessarily the most likely one; always read the whole offered distribution.

Evaluation

Frozen model, saved per-paper predictions. Higher is better except calibration error. Human-label tests and reference-model-label tests are distinguished. No claim that benchmark differences establish universal superiority. The exact aggregate scores are also supplied in benchmarks.json.

Original expert benchmark: 200 articles, 176 Science-Metrix subfields

Agreement with the experts' modal first choice; ties accept any tied label. Each half has only 100 articles, so small differences are uncertain. Source: Rivest et al..

Method	Specialist journals	General journals	Overall
This 2B release	57%	66%	61.5%
Jev 1.13, flat, same test texts	63%	62%	62.5%
Qwen 27B, direct (historical run)	57%	64%	60.5%
Qwen 27B, thinking (historical run)	64%	67%	65.5%

Models use their native scoring interfaces and prompts; the Qwen runs are historical, not a new matched latency benchmark. For this release, top-3 contains an expert-mode subfield on 86.0% of articles; option-order stability is 76.5%; confidence ECE is 0.0460. These are different properties, not substitutes for accuracy.

Larger held-out-paper evaluations

Test	This 2B release
arXiv author primary category (2,980 papers, 149 options)	57.3%
arXiv any listed category	70.4%
arXiv title only	46.9%
arXiv 8 groups, asked directly	80.6%
arXiv same pick after shuffling	81.8%
MeSH centrality proxy: major vs other headings, AUC	0.8372
MeSH major vs minor headings, AUC	0.6069
Eight reserved topic-membership tasks, macro average precision	0.8196

arXiv measures agreement with authors' category conventions; the papers are held back, but the taxonomy is not necessarily unseen. The MeSH benchmark uses indexer major-topic flags: minor headings can still describe a real subject of the paper, and publication year alone does not prove every record was manually indexed. Treat MeSH as an indexing-convention proxy, not perfect ground truth for the broader word "central". The eight-topic test uses reference-model labels and retrieval-population weights, not human gold.

Original reading-list topic judgment: regressions remain

These are reused development topics, judged against reference-model labels. Precision/recall use a fixed 0.5 threshold, not a threshold fitted on the test. They measure the judge alone, not a complete map/reading-list replay.

Topic	Precision	Recall	F1	Ranking AUC
Habitat fragmentation	0.708	0.808	0.755	0.9702
LLM post-training	0.510	0.940	0.662	0.9882

Better taxonomy scores did not consistently improve the original reading-list task. Earlier topic-only models ranked fragmentation papers better, and Jev remains stronger on these original topic-membership comparisons. Do not deploy this version as an automatic upgrade for a topic judge without task-specific validation.

Limitations and responsible use

• Prompt-format dependence: on the 0.8B release, moving a long taxonomy after the paper reduced arXiv accuracy from 54.1% to 5.9%. That specific order-swap measurement is for the 0.8B; the same number is not asserted for 2B. Use the helper for either size. Experimental mixed-order repair models are not these released weights.
• Options compete: neighboring disciplines, method versus application, and catch-all categories are ambiguous. Broad-field selection is not uniformly reliable across taxonomies.
• English scientific titles/abstracts are the primary use case. Title-only inputs, sentences and technical methods paragraphs are weaker. A paragraph's parent-paper label is only a proxy for its own topic.
• News, sport, general web pages and other non-research text can receive confident scientific classifications. "None of these" is not a reliable universal rejection mechanism. Broader arbitrary-text use is unvalidated.
• Confidence depends on the option set, input domain and task. Low ECE on one benchmark does not establish universal calibration. Validate thresholds on independent data for your use case.
• Numeric results can differ slightly across Transformers/vLLM, batch layouts and BF16 kernels, particularly for near-tied options. Exact prompt token construction is more important than rounding the probabilities.
• Do not use alone for clinical decisions, research-quality judgments, employment decisions or other consequential assessments. A scientific field label is not a measure of credibility, safety or value.
• The training-data construction is not disclosed; independent retraining from the published files alone is not possible. Model and helper inference are independently runnable.

Files and compatibility

• model.safetensors: standalone BF16 weights; no adapter merge required.
• config.json, tokenizer files and chat template: load with the tested Transformers stack.
• classifier.json: exact codebook, answer token IDs and list-first format contract.
• paper_classifier.py: Transformers helper and shared prompt builder.
• vllm_fast.py: prefix-cached bulk helper.
• benchmarks.json: aggregate evaluations, without source paper texts or private data-generation details.
• release.json: weight SHA-256, exact stored parameter count, file size and tested package versions.
• LICENSE: Apache 2.0.

Tested with Transformers 5.3.0, PyTorch 2.11.0 and vLLM 0.29.0. Newer versions may work but are not covered by these checks. On CUDA, optional accelerated linear-attention/convolution libraries may affect speed; no speed guarantee is made for an untested environment. The original public model repositories remain available and unchanged.