Anders-sonderby/patentsbert-finetuned

Model Card: PatentSBERTa Fine-Tuned on Green Patent Claims (Assignment 2)

Model Summary

This model is a fine-tuned version of AI-Growth-Lab/PatentSBERTa for binary classification of patent claims as green technology (Y02) or not. It was developed as part of a course assignment in Applied Deep Learning and AI at Aalborg University, exploring Human-in-the-Loop (HITL) data labeling pipelines for patent classification.

---

Model Details

• Developed by: Anders Sønderbý (as58zr@student.aau.dk)
• Model type: Sentence Transformer with classification head (binary)
• Base model: AI-Growth-Lab/PatentSBERTa
• Language: English
• License: MIT
• Task: Binary text classification — Green Technology (Y02) vs. Not Green

---

What This Model Does

Given the text of a patent claim, the model predicts whether the claim relates to green technology as defined by the CPC Y02 classification system. The output is a binary label:

• 1 — Green technology (Y02)
• 0 — Not green technology

---

Training Pipeline Overview

This model was produced through a 4-stage pipeline:

Stage 1 — Baseline (Frozen Embeddings)

A baseline classifier was trained using frozen PatentSBERTa embeddings with a Logistic Regression head on the train_silver split. This baseline was used to compute uncertainty scores for active learning.

Stage 2 — Uncertainty Sampling

The baseline model computed p_green (predicted probability of green) for all examples in pool_unlabeled. An uncertainty score was computed as:

u = 1 − 2 · |p − 0.5|

The top 100 highest-uncertainty claims were exported as hitl_green_100.csv for human review.

Stage 3 — LLM → Human HITL Labeling

For each of the 100 high-risk claims, an LLM first suggested a label (llm_green_suggested), confidence (llm_confidence), and rationale (llm_rationale). A human reviewer then assigned the final gold label (is_green_human), overriding the LLM where necessary. Only the claim text was used during labeling — no CPC codes or metadata.

Stage 4 — Fine-Tuning

PatentSBERTa was fine-tuned for binary classification using the combined train_silver + gold_100 dataset, where gold labels override silver labels for the 100 HITL-reviewed claims.

---

Training Data

• Dataset: Derived from AI-Growth-Lab/patents_claims_1.5m_traim_test
• Working file: patents_50k_green.parquet — a balanced 50k sample (25,000 green, 25,000 not green)
• Silver label source: CPC Y02* classification codes (is_green_silver)
• Gold labels: 100 human-reviewed claims from uncertainty sampling (is_green_gold)

Dataset Splits

Split	Size	Description
train_silver	~40,000	Silver-labeled training set (CPC-derived)
eval_silver	~5,000	Silver-labeled evaluation set
pool_unlabeled	~5,000	Unlabeled pool used for uncertainty sampling
gold_100	100	Human-reviewed high-uncertainty claims

---

Training Hyperparameters

Parameter	Value
Base model	AI-Growth-Lab/PatentSBERTa
Max sequence length	256
Epochs	1
Learning rate	2e-5
Training set size	~40,100 (train_silver + gold_100)

---

Evaluation Results

Evaluation Set	F1 Score	Notes
eval_silver (5,000)	0.818	Primary evaluation metric
gold_100 (100)	0.667	Human-reviewed high-uncertainty claims

---

HITL Reporting

As required by the assignment, the human reviewer assessed all 100 high-uncertainty claims. The LLM suggestion and human final label were recorded for each claim. Disagreements between the LLM and human reviewer were documented in the labeling file (hitl_final.csv).

---

Intended Use

• Primary use: Academic research and coursework in patent classification
• Intended users: Course instructors and students at Aalborg University
• Out-of-scope: Production patent classification systems, legal patent assessment, or any commercial use

---

Limitations

• Trained on a balanced 50k sample — performance may differ on the full unbalanced patent corpus
• Silver labels are derived from CPC codes, which may contain noise
• The model classifies based on claim text only and does not use metadata, citations, or CPC codes at inference time
• Fine-tuned for 1 epoch only due to compute constraints

---

Repository

The full code, notebooks, and data files for this assignment are available in the course GitHub repository.