Buckets:
| dataset_info: | |
| features: | |
| - name: cik | |
| dtype: string | |
| - name: company | |
| dtype: string | |
| - name: year | |
| dtype: int64 | |
| - name: filing_date | |
| dtype: string | |
| - name: sic | |
| dtype: string | |
| - name: state_of_inc | |
| dtype: string | |
| - name: filing_html_index | |
| dtype: string | |
| - name: accession_number | |
| dtype: string | |
| - name: table_image | |
| dtype: image | |
| - name: table_body | |
| dtype: string | |
| - name: executives | |
| dtype: string | |
| - name: sct_probability | |
| dtype: float64 | |
| splits: | |
| - name: train | |
| num_examples: 12022 | |
| - name: year_2005 | |
| num_examples: 468 | |
| - name: year_2006 | |
| num_examples: 492 | |
| - name: year_2007 | |
| num_examples: 512 | |
| - name: year_2008 | |
| num_examples: 486 | |
| - name: year_2009 | |
| num_examples: 472 | |
| - name: year_2010 | |
| num_examples: 429 | |
| - name: year_2011 | |
| num_examples: 434 | |
| - name: year_2012 | |
| num_examples: 391 | |
| - name: year_2013 | |
| num_examples: 384 | |
| - name: year_2014 | |
| num_examples: 430 | |
| - name: year_2015 | |
| num_examples: 434 | |
| - name: year_2016 | |
| num_examples: 357 | |
| - name: year_2017 | |
| num_examples: 369 | |
| - name: year_2018 | |
| num_examples: 374 | |
| - name: year_2019 | |
| num_examples: 344 | |
| - name: year_2020 | |
| num_examples: 4867 | |
| - name: year_2021 | |
| num_examples: 393 | |
| - name: year_2022 | |
| num_examples: 386 | |
| configs: | |
| - config_name: default | |
| data_files: | |
| - split: train | |
| path: data/train-* | |
| - split: year_2005 | |
| path: data/year_2005-* | |
| - split: year_2006 | |
| path: data/year_2006-* | |
| - split: year_2007 | |
| path: data/year_2007-* | |
| - split: year_2008 | |
| path: data/year_2008-* | |
| - split: year_2009 | |
| path: data/year_2009-* | |
| - split: year_2010 | |
| path: data/year_2010-* | |
| - split: year_2011 | |
| path: data/year_2011-* | |
| - split: year_2012 | |
| path: data/year_2012-* | |
| - split: year_2013 | |
| path: data/year_2013-* | |
| - split: year_2014 | |
| path: data/year_2014-* | |
| - split: year_2015 | |
| path: data/year_2015-* | |
| - split: year_2016 | |
| path: data/year_2016-* | |
| - split: year_2017 | |
| path: data/year_2017-* | |
| - split: year_2018 | |
| path: data/year_2018-* | |
| - split: year_2019 | |
| path: data/year_2019-* | |
| - split: year_2020 | |
| path: data/year_2020-* | |
| - split: year_2021 | |
| path: data/year_2021-* | |
| - split: year_2022 | |
| path: data/year_2022-* | |
| license: mit | |
| task_categories: | |
| - table-to-text | |
| language: | |
| - en | |
| tags: | |
| - finance | |
| - sec | |
| - executive-compensation | |
| - def14a | |
| - proxy-statements | |
| pretty_name: SEC Executive Compensation | |
| size_categories: | |
| - 10K<n<100K | |
| # SEC Executive Compensation Dataset | |
| > [!CAUTION] | |
| > ## ๐ง DATASET UNDER CONSTRUCTION ๐ง | |
| > | |
| > This dataset is actively being developed and expanded. The current version contains **~12,000 records** out of a target of **100,000+ SEC filings** (2005-2022). | |
| > | |
| > **What to expect:** | |
| > - Data may contain errors or inconsistencies | |
| > - Schema and fields may change | |
| > - More records will be added regularly | |
| > - Statistics will be updated as processing continues | |
| > | |
| > **Use at your own risk for research purposes only.** | |
| ๐ **Pipeline**: [github.com/pierpierpy/Execcomp-AI](https://github.com/pierpierpy/Execcomp-AI) | |
| Structured executive compensation data extracted from SEC DEF 14A proxy statements using AI. | |
| ```python | |
| from datasets import load_dataset | |
| # Load all data | |
| ds = load_dataset("pierjoe/execcomp-ai-sample", split="train") | |
| # Load specific year only | |
| ds_2020 = load_dataset("pierjoe/execcomp-ai-sample", split="year_2020") | |
| ``` | |
| <details> | |
| <summary><b># ๐ Dataset Statistics</b> (click to expand)</summary> | |
|  | |
|  | |
|  | |
| </details> | |
| --- | |
| <details> | |
| <summary><b>๐ฏ SCT Probability & Quality</b> (click to expand)</summary> | |
| ### Why `sct_probability`? | |
| The main VLM (Qwen3-32B) sometimes: | |
| 1. **False positives**: Classifies non-SCT tables as SCT (e.g., Director Compensation tables) | |
| 2. **Duplicates**: Some documents have multiple tables classified as SCT when only one is the real Summary Compensation Table | |
| A **fine-tuned binary classifier** (Qwen3-VL-4B) scores each table with a probability (0-1) to help filter these cases. | |
|  | |
|  | |
| > ๐ก **Recommendation**: Filter by `sct_probability >= 0.7` to get high-confidence records only. | |
| </details> | |
| --- | |
| <details> | |
| <summary><b>๐ฐ Compensation Statistics</b> (click to expand)</summary> | |
|  | |
|  | |
| ### ๐ Top 10 Highest Paid Executives | |
|  | |
| ### Compensation Distribution | |
|  | |
| ### Trends Over Time | |
|  | |
| </details> | |
| --- | |
| ## ๐ Dataset Description | |
| This dataset contains **Summary Compensation Tables** extracted from SEC filings, with: | |
| - Original table images | |
| - HTML table structure | |
| - Structured JSON with executive compensation details | |
| - **SCT probability score** from a fine-tuned binary classifier (to filter false positives) | |
| > ๐ก **Tip**: Filter by `sct_probability >= 0.7` to get high-confidence SCT records only. | |
| ## Fields | |
| | Field | Type | Description | | |
| |-------|------|-------------| | |
| | `cik` | string | SEC Central Index Key | | |
| | `company` | string | Company name | | |
| | `year` | int | Filing year | | |
| | `filing_date` | string | SEC filing date | | |
| | `sic` | string | Standard Industrial Classification code | | |
| | `state_of_inc` | string | State of incorporation | | |
| | `filing_html_index` | string | Link to SEC filing | | |
| | `accession_number` | string | SEC accession number | | |
| | `table_image` | image | Extracted table image | | |
| | `table_body` | string | HTML table content | | |
| | `executives` | string | JSON array of executive compensation | | |
| | `sct_probability` | float | Probability (0-1) that this is a real SCT (from fine-tuned classifier) | | |
| ## Executive Schema | |
| ```json | |
| { | |
| "name": "John Smith", | |
| "title": "CEO", | |
| "fiscal_year": 2023, | |
| "salary": 500000, | |
| "bonus": 100000, | |
| "stock_awards": 2000000, | |
| "option_awards": 500000, | |
| "non_equity_incentive": 300000, | |
| "change_in_pension": 50000, | |
| "other_compensation": 25000, | |
| "total": 3475000 | |
| } | |
| ``` | |
| --- | |
| ## ๐ Quick Start | |
| ```python | |
| from datasets import load_dataset | |
| import json | |
| # Load full dataset | |
| ds = load_dataset("pierjoe/execcomp-ai-sample", split="train") | |
| # Or load a specific year | |
| ds_2020 = load_dataset("pierjoe/execcomp-ai-sample", split="year_2020") | |
| # Filter by SCT probability (recommended to reduce false positives) | |
| ds_filtered = ds.filter(lambda x: x["sct_probability"] >= 0.7) | |
| # View a record | |
| rec = ds_filtered[0] | |
| print(rec["filing_html_index"]) | |
| display(rec['table_image']) # PIL Image | |
| print(json.dumps(json.loads(rec['executives']), indent=2)) | |
| ``` | |
| ### Analyze with Pandas | |
| ```python | |
| import pandas as pd | |
| import json | |
| # Convert to DataFrame | |
| df = ds_filtered.to_pandas() | |
| # Parse all executives | |
| all_execs = [] | |
| for _, row in df.iterrows(): | |
| for exec_data in json.loads(row['executives']): | |
| exec_data['company'] = row['company'] | |
| exec_data['year'] = row['year'] | |
| all_execs.append(exec_data) | |
| exec_df = pd.DataFrame(all_execs) | |
| # Average compensation by year | |
| print(exec_df.groupby('year')['total'].mean()) | |
| # Top 10 highest paid | |
| print(exec_df.nlargest(10, 'total')[['name', 'company', 'year', 'total']]) | |
| ``` | |
| ### View Table Image | |
| ```python | |
| # Display table image | |
| record = ds["train"][0] | |
| record["table_image"] # PIL Image object | |
| ``` | |
| --- | |
| ## ๐ง Source & Methodology | |
| Data extracted from [SEC EDGAR](https://www.sec.gov/edgar) DEF 14A filings using: | |
| - **[MinerU](https://github.com/opendatalab/MinerU)** for PDF table extraction | |
| - **[Qwen3-VL-32B](https://huggingface.co/Qwen/Qwen3-VL-32B-Instruct)** for classification and extraction | |
| ### Pipeline Steps | |
| 1. Download DEF 14A PDFs from SEC EDGAR | |
| 2. Extract tables with MinerU (VLM-based) | |
| 3. Classify tables to identify Summary Compensation Tables | |
| 4. Merge tables split across pages | |
| 5. Extract structured compensation data with VLM | |
| --- | |
| ## ๐ License | |
| MIT License | |
| --- | |
| ## ๐ Citation | |
| If you use this dataset in your research, please cite: | |
| ```bibtex | |
| @dataset{execcomp_ai_2026, | |
| author = {Di Pasquale, Piergiorgio}, | |
| title = {SEC Executive Compensation Dataset}, | |
| year = {2026}, | |
| publisher = {Hugging Face}, | |
| url = {https://huggingface.co/datasets/pierjoe/execcomp-ai-sample}, | |
| note = {AI-extracted executive compensation data from SEC DEF 14A filings (2005-2022)} | |
| } | |
| ``` | |
| Or in text format: | |
| > Di Pasquale, P. (2026). *SEC Executive Compensation Dataset*. Hugging Face. https://huggingface.co/datasets/pierjoe/execcomp-ai | |
| ## ๐ Links | |
| - **GitHub**: [github.com/pierpierpy/Execcomp-AI](https://github.com/pierpierpy/Execcomp-AI) | |
| - **SEC EDGAR**: [sec.gov/edgar](https://www.sec.gov/edgar) | |
Xet Storage Details
- Size:
- 8.86 kB
- Xet hash:
- f258844080c0c17c54b51c20da098793f64a65efafc41f6ced04824a933f8ffb
ยท
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.