\documentclass[11pt,a4paper]{article}
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\usepackage[margin=24mm]{geometry}
\usepackage{microtype}
\usepackage{booktabs,longtable,tabularx,array}
\usepackage{xcolor,graphicx,float}
\usepackage[hidelinks]{hyperref}
\usepackage{xurl}
\usepackage{enumitem}
\definecolor{navy}{HTML}{17365D}
\definecolor{lightblue}{HTML}{EAF2F8}
\definecolor{darkgray}{HTML}{4A4A4A}
\hypersetup{pdftitle={PATSTAT AI Complete Master Technical Report},pdfauthor={STPI AI Patent Data Project}}
\setlength{\parindent}{0pt}
\setlength{\parskip}{5pt}
\renewcommand{\arraystretch}{1.12}
\setlength{\emergencystretch}{2em}
\newcommand{\pct}{\%}

\begin{document}

\begin{titlepage}
\centering
\vspace*{28mm}
{\color{navy}\rule{\textwidth}{1.2pt}}\\[12mm]
{\Huge\bfseries PATSTAT AI Complete Master\\[3mm]Technical Report\par}
\vspace{8mm}
{\Large Legacy-priority integration, country completion, and IPC/CPC backfill\par}
\vspace{16mm}
{\large Dataset snapshot: 12 July 2026\par}
{\large Source release: PATSTAT Online 2026 Spring\par}
\vfill
\begin{tabular}{rl}
Observation grain: & One row per PATSTAT application identifier (\texttt{app\_id})\\
Rows: & 2,330,553\\
Columns: & 175\\
Coverage years: & 1950--2026\\
\end{tabular}
\vfill
{\color{navy}\rule{\textwidth}{1.2pt}}
\end{titlepage}

\pagenumbering{roman}
\tableofcontents
\newpage
\pagenumbering{arabic}

\section{Technical summary}

The final PATSTAT AI Complete Master contains \textbf{2,330,553 unique patent applications and 175 columns}. Full-file validation found exactly 2,330,553 distinct \texttt{app\_id} values, zero duplicated application identifiers, and a valid GZIP stream. The intended analytical grain is one row per PATSTAT application.

The master combines a 2,037,379-row legacy Stage3 AI dataset with 479,889 unique applications collected from PATSTAT Online 2026 Spring for 2023--2026. A total of 186,715 newly collected applications overlapped the legacy dataset; shared nonblank legacy values were retained. The integration appended 293,174 new-only applications and added 19 fields that were absent from the legacy schema.

A subsequent combined PATSTAT classification query collected the application identifier, priority year, and aggregated IPC and CPC code lists for every year from 1950 through 2026. IPC coverage increased to 2,326,676 applications (99.83\%). CPC coverage is 1,850,180 applications (79.39\%); the remaining CPC gaps reflect applications without an observed PATSTAT CPC relation rather than incomplete annual downloads.

Country participation flags are complete as stored variables, but their interpretation depends on the ordered completion rule. Direct applicant and inventor countries take precedence, followed by patent-family evidence, organization-name matching, and finally KR/US/CN filing-office fallback. Filing-office fallback is an operational participation assignment and does not establish inventor nationality or residence.

\section{Dataset scope and lineage}

\subsection{Input datasets}

\begin{tabularx}{\textwidth}{>{\raggedright\arraybackslash}Xrr>{\raggedright\arraybackslash}X}
\toprule
Input & Rows & Columns & Role \\
\midrule
Legacy Stage3 AI master & 2,037,379 & 156 & Existing classified AI corpus and standardized person fields \\
New PATSTAT 2023--2026 collection & 479,889 & 51 & Bibliographic, person, IPC, CPC, publication, and priority enrichment \\
PATSTAT annual classification backfill & 6,186,011 app IDs observed & 4 & IPC/CPC completion for 1950--2026 \\
\bottomrule
\end{tabularx}

The annual backfill population is larger than the final AI master because the PATSTAT query returns the full AI-filtered candidate population for each release-year interval. Only matching final-master \texttt{app\_id} values are applied.

\subsection{Final row composition}

\begin{table}[H]
\centering
\begin{tabular}{lr}
\toprule
Record provenance & Rows \\
\midrule
Legacy only & 1,850,664 \\
Legacy/new overlap, legacy priority & 186,715 \\
New only & 293,174 \\
\midrule
\textbf{Final unique applications} & \textbf{2,330,553} \\
\bottomrule
\end{tabular}
\end{table}

\section{Collection and integration methodology}

\subsection{Application-level integration}

PATSTAT relation tables were first aggregated to application grain. Applicant and inventor records were linked through PATSTAT person-application relations. IPC, CPC, publication, and priority relations were deduplicated within each application and stored as semicolon-separated lists. The final integration uses \texttt{app\_id} as the primary key.

The merge rule is:
\begin{enumerate}[leftmargin=*]
\item Retain each legacy row and every nonblank legacy value.
\item For an overlapping \texttt{app\_id}, fill a selected shared field only when the legacy value is blank.
\item Add columns present only in the new collection.
\item Append new-only \texttt{app\_id} values.
\item Revalidate unique application grain after every material rewrite.
\end{enumerate}

\subsection{Combined IPC/CPC backfill}

The final classification query returns one row per application:
\begin{verbatim}
app_id | priority_year | all_ipc_codes | all_cpc_codes
\end{verbatim}

IPC and CPC are aggregated independently with \texttt{STRING\_AGG} in correlated subqueries. This avoids an IPC-by-CPC Cartesian product. Annual queries were executed for all 77 years from 1950 through 2026. A total of 661 validated CSV download parts were produced. Each part was checked for required headers and expected row count before being accepted.

\section{Country participation completion}

Country flags follow this deterministic hierarchy:
\begin{enumerate}[leftmargin=*]
\item standardized applicant-country codes;
\item standardized inventor-country codes when applicant country is unavailable;
\item union of DOCDB family countries, then INPADOC family countries;
\item boundary-aware standardized organization-name matching;
\item KR, US, or CN filing-office fallback;
\item explicit \texttt{none} when no rule applies.
\end{enumerate}

\begin{table}[H]
\centering
\begin{tabular}{lr}
\toprule
Final fill method & Applications \\
\midrule
Applicant & 982,860 \\
Inventor & 6,498 \\
Family & 169,352 \\
Organization name & 64,529 \\
Filing office & 980,540 \\
None & 126,774 \\
\bottomrule
\end{tabular}
\end{table}

Final country-participation totals are KR 170,936, US 481,680, and CN 1,105,478. These totals overlap because one application can carry more than one country flag.

\section{Column descriptive statistics}

\subsection{Completeness distribution}

\begin{table}[H]
\centering
\begin{tabular}{lr}
\toprule
Completeness interval & Number of columns \\
\midrule
100\% & 31 \\
95\% to $<$100\% & 5 \\
80\% to $<$95\% & 98 \\
50\% to $<$80\% & 6 \\
1\% to $<$50\% & 30 \\
$>$0\% to $<$1\% & 4 \\
0\% & 1 \\
\midrule
Total & 175 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Core bibliographic and classification fields}

\begin{longtable}{p{0.38\textwidth}rrr}
\toprule
Column & Nonblank & Missing & Coverage \\
\midrule
\endhead
\texttt{app\_id} & 2,330,553 & 0 & 100.00\% \\
\texttt{patent\_office} & 2,330,553 & 0 & 100.00\% \\
\texttt{priority\_year} & 2,330,553 & 0 & 100.00\% \\
\texttt{title} & 2,325,259 & 5,294 & 99.77\% \\
\texttt{abstract} & 2,259,040 & 71,513 & 96.93\% \\
\texttt{all\_ipc\_codes} & 2,326,676 & 3,877 & 99.83\% \\
\texttt{ipc\_count} & 2,326,676 & 3,877 & 99.83\% \\
\texttt{all\_cpc\_codes} & 1,850,180 & 480,373 & 79.39\% \\
\texttt{cpc\_count} & 1,850,180 & 480,373 & 79.39\% \\
\bottomrule
\end{longtable}

\subsection{Applicant and inventor fields}

\begin{longtable}{p{0.42\textwidth}rrr}
\toprule
Column & Nonblank & Missing & Coverage \\
\midrule
\endhead
\texttt{stage3\_applicant\_names\_std} & 1,992,673 & 337,880 & 85.50\% \\
\texttt{stage3\_applicant\_countries} & 1,992,674 & 337,879 & 85.50\% \\
\texttt{stage3\_applicant\_affiliation} & 693,818 & 1,636,735 & 29.77\% \\
\texttt{stage3\_inventor\_names\_std} & 1,982,429 & 348,124 & 85.06\% \\
\texttt{stage3\_inventor\_countries} & 1,982,429 & 348,124 & 85.06\% \\
\texttt{stage3\_inventor\_affiliation} & 1,473,896 & 856,657 & 63.24\% \\
\texttt{applicant\_info} & 466,084 & 1,864,469 & 20.00\% \\
\texttt{inventor\_info} & 464,579 & 1,865,974 & 19.93\% \\
\bottomrule
\end{longtable}

The nonblank rate for country strings is not the same as the valid ISO-country rate because strings can contain \texttt{Unknown}. Country-level inference must use parsed valid codes and explicitly report the denominator.

\section{Quality assurance results}

\begin{table}[H]
\centering
\begin{tabularx}{\textwidth}{Xr}
\toprule
Check & Result \\
\midrule
Final row count & 2,330,553 \\
Final column count & 175 \\
Distinct \texttt{app\_id} & 2,330,553 \\
Duplicate \texttt{app\_id} rows & 0 \\
Blank \texttt{country\_fill\_method} & 0 \\
IPC count/list mismatches & 0 \\
CPC count/list mismatches & 0 \\
Annual classification years completed & 77 of 77 \\
Classification download parts & 661 \\
GZIP integrity & Passed \\
\bottomrule
\end{tabularx}
\end{table}

The only entirely blank column is \texttt{inventor\_addresses}. It remains blank because the newly collected PATSTAT person rows did not contain nonblank inventor-address values; no address was inferred from another field.

\section{Limitations and interpretation}

\begin{itemize}[leftmargin=*]
\item Filing office and inventor country are different concepts. Office fallback is retained as a documented operational rule, not a nationality measure.
\item Multinational applications contribute to every represented country; country totals therefore overlap.
\item Person-country coverage differs substantially across patent offices, so inventor-country comparisons describe observed records rather than a complete population.
\item CPC coverage is lower than IPC coverage because not every PATSTAT application has an observed CPC relation.
\item The 2026 partition contains only nine applications in the current collection and is not a complete calendar-year total.
\item AI taxonomy fields are populated primarily for the legacy 2,037,379-row corpus. Newly appended applications can have blank historical AI-category columns even though they were selected by the PATSTAT AI query policy.
\end{itemize}

\section{Reproducibility and deliverables}

Primary data file:
\begin{quote}\small
\path{patstat_ai_complete_master_legacy_priority_20260712.csv.gz}
\end{quote}

Supporting artifacts include the machine-readable integration and backfill reports, full 175-column descriptive-statistics CSV, annual collection status, SHA-256 checksum, and the Python scripts used for integration, country completion, blank-field coalescing, and IPC/CPC application.

The validated data-file SHA-256 at report generation time is:
\begin{quote}\ttfamily\small
00f48da33ab6064f0fad4c891e8bc6aecc86f6ccde860767e1927777ee9fe43b
\end{quote}

\end{document}
