moosairfaan/vol10k-tfidf-logreg

vol10k TF-IDF logistic model

Predictions estimate the preregistered direction of post-filing realized volatility: whether volatility over the next 63 close-to-close returns is strictly higher than volatility over the preceding 63. They are not investment advice, not a return forecast, and not a trading signal.

This repository is the published release. The code and model artifacts are under the Apache License, Version 2.0. See LICENSE.txt. That license does not cover EDGAR filings or Yahoo market data. See DATA_RIGHTS.md.

Task

The input is Item 1A and Item 7 from an original Form 10-K, plus the sealed numeric features. The output is a probability for label_primary.

The primary label clock ends the pre-window on the NYSE close of the acceptance date, or on the previous session when that date is closed, even if the filing was accepted before the close. The post-window is the next 63 session returns. Realized volatility is the sample standard deviation of those close-to-close log returns, divisor n - 1, not annualized. The label is 1 only when post-window volatility is strictly greater. Prices used to build labels were split-adjusted and were not dividend-adjusted. Feature prices stop at the last scheduled close at or before acceptance.

Data scope

The fit is the approved 20-ticker slice of a current S&P 500 snapshot, so companies that left the index are absent. Tickers: JPM, FITB, PGR, PLD, NEE, AES, MSFT, GOOG, PLTR, META, AAPL, XOM, CAT, HD, KO, RTX, HRL, IEX, CHRW, NDSN. Form 10-K only, not 10-K/A. Acceptance dates from 2010-01-01 in America/New_York.

The fitted model and the scored comparison use complete cases. A filing is excluded from every model when Item 1A or Item 7 is empty. PREREGISTRATION.md does not seal that rule. Phase 6a left missing Item 7 unresolved. Phase 6b applied the complete-case rule, and the holdout used the same rule. Of 225 development filings, 57 were dropped and 168 were kept. Three of the 168 have a label horizon that reaches 2023, so the training set is 165. The same fit scores 2023 and 2024. 2024 was not refit on 2023. This release documents that sample. It does not refit it.

Features

Text is Item 1A followed by Item 7, joined by a newline. The vectorizer uses unigrams and bigrams, min_df 5, 30,000 features, sublinear term frequency, lowercase, and no stop-word list. It was fit on the 165 training filings only. Training text had the issuer's snapshot name and issuer-specific aliases replaced with [COMPANY], and contextual ticker mentions replaced with [TICKER]. This release does not include the filing parser. Callers supply the two section strings.

Numeric columns, median-imputed and scaled on the training rows, with a missingness indicator from the imputer and the study's own missingness columns: trailing_rv, market_rv, size, log_mkt_cap, log_assets, log_revenue, log1p_long_term_debt, liabilities_to_assets, equity_to_assets, net_income_to_assets, operating_income_to_assets, cash_to_assets, revenue_growth, and each *_missing column listed in config.json. sector is one-hot encoded. Unknown sectors are ignored at inference.

Training configuration

Seed 42. L2 logistic regression, solver lbfgs, maximum 2,000 iterations. C is 0.01, the value selected on the 2022 inner-validation slice of the purged training set by Brier score before the holdout was scored. This release did not choose C again. Scientific config sha256: 86859a8611299150bc0e9f872973c2629ffce770b360962b2ea0eb5adcc769fe. The weight file's sha256 is 068fa214f39dd7f4883548744832d263808d47f4fa1deee0dca23a2535a09d63.

Python 3.12. Pins are in requirements.txt: numpy 2.5.3, pandas 3.0.6, scikit-learn 1.9.1, joblib 1.6.0.

Validation

Development walk-forward, test years 2015–2022, 116 filings, positive rate 0.474. This is not the claim.

Model	AUC	Brier
Majority	0.500	0.260
Numeric logistic regression	0.604	0.286
Numeric gradient boosting	0.666	0.275
TF-IDF plus logistic regression	0.611	0.276

Delta AUC versus numeric logistic regression was +0.006. Delta Brier was +0.010. Versus numeric gradient boosting, delta AUC was −0.056.

Holdout

One score, run 20260930T220553Z_holdout. 34 filings, 17 issuers, positive rate 0.382. Four holdout filings were dropped, JPM and PGR in 2023 and 2024. On each, Item 1A was extracted and Item 7 was an incorporation pointer under 500 characters.

Model	AUC	Brier
Majority (training positive rate 0.467; AUC defined as 0.5)	0.500	0.243
Numeric logistic regression	0.630	0.233
Numeric gradient boosting	0.568	0.291
TF-IDF plus logistic regression	0.630	0.233

Delta AUC versus numeric logistic regression is exactly 0. Delta Brier is +0.000130. The two probability vectors are not identical: all 34 pairs differ, and the largest absolute gap is 0.00296. The holdout report prints the Brier delta as 0.000 because it rounds to three decimals. In 2023 both logistic models have AUC 0.485 (17 filings). In 2024 both have AUC 0.700 (17 filings).

The confirmatory interval is a firm-clustered bootstrap: resample CIKs with replacement, 1,000 replicates, seed 42, percentiles 2.5 and 97.5. Replicates with fewer than two classes are invalid. The interval for delta AUC versus numeric logistic regression is 0.000 to 0.000 (1,000 valid replicates). It does not lie entirely above zero, so the preregistered rule reports no confirmed incremental text value.

The year-quarter block bootstrap is diagnostic only. Its interval for the same delta is 0.000 to 0.000 (997 valid replicates). Versus numeric gradient boosting, delta AUC is +0.062. The firm interval is −0.110 to 0.245 and the year-quarter interval is −0.230 to 0.214. Those intervals do not decide the claim.

Inference

From this directory, with the pinned packages installed:

python predict.py example_input.json
python check_release.py

example_input.json is synthetic. It is not a filing, a price, or a label. python predict.py example_input.json prints 0.4382761327.

predict.py joins Item 1A and Item 7 with a newline, applies the saved vectorizer and numeric preprocessor, and returns the positive-class probability.

Limitations

The sample is a 20-company slice of today's index, not the full S&P 500 and not the companies that left. Complete-case exclusion removes issuers whose Item 7 is incorporated by reference, including every JPM and PGR filing in this pilot. The firm-clustered interval of 0.000 to 0.000 is the AUC difference against numeric logistic regression. It is not evidence that the probabilities are equal, and it does not lie entirely above zero, so there is no confirmed incremental text value. Thirty-one of the 34 holdout probabilities from this model sit between 0.4 and 0.6. The firm interval treats issuers as independent even when they file in the same quarter. The year-quarter interval has six blocks, most of them in the first quarter, and treats quarters as independent. Development scores are not the claim. No decision threshold was fit. config.json records holdout_probability_max_abs_diff of 0 because the saved weights match the stored TF-IDF probabilities, not because those probabilities match numeric logistic regression.

Intended use

Use the probability to reproduce or inspect this preregistered volatility-direction task. Do not use it as investment advice, as a portfolio signal, or as a description of any issuer outside the fitted sample without a new study.